AWS Presigned URL 与 Python 的 Requests 库一起工作，但在 cURL 下失败

Question

最近我开始使用 AWS 预签名 URLs 将文件上传到 S3。使用 Python 的 Requests 库时，生成的预签名 URL 工作正常，如下所示：

生成预签名 url:

def create_presigned_post(bucket_name, object_name,
                          fields=None, conditions=None, expiration=3600):
    """Generate a presigned URL S3 POST request to upload a file

    :param bucket_name: string
    :param object_name: string
    :param fields: Dictionary of prefilled form fields
    :param conditions: List of conditions to include in the policy
    :param expiration: Time in seconds for the presigned URL to remain valid
    :return: Dictionary with the following keys:
        url: URL to post to
        fields: Dictionary of form fields and values to submit with the POST
    :return: None if error.
    """

    # Generate a presigned S3 POST URL
    s3_client = boto3.client('s3')
    try:
        response = s3_client.generate_presigned_post(bucket_name,
                                                     object_name,
                                                     Fields=fields,
                                                     Conditions=conditions,
                                                     ExpiresIn=expiration)
    except ClientError as e:
        logging.error(e)
        return None

    # The response contains the presigned URL and required fields
    return response

运行获取预签名的请求 url


# Getting a presigned_url to upload the file into S3 Bucket.
        headers = {'Content-type': 'application/json', 'request': 'upload_url', 'target': FILENAME, 'x-api-key': API_KEY}

        r_upload = requests.post(url = API_ENDPOINT, headers = headers)

        url = json.loads(json.loads(r_upload.text)['body'])['url']
        fields_ = json.loads(json.loads(r_upload.text)['body'])['fields']
        fields = {
                "x-amz-algorithm": fields_["x-amz-algorithm"],
                "key": fields_["key"],
                "policy": fields_["policy"],
                "x-amz-signature": fields_["x-amz-signature"],
                "x-amz-date": fields_["x-amz-date"],
                "x-amz-credential": fields_["x-amz-credential"],
                "x-amz-security-token":  fields_["x-amz-security-token"]
        }

        fileobj = open(FILENAME, 'rb')
        http_response = requests.post(url, data=fields,files={'file': (FILENAME, fileobj)})

有效回复

 "{\"url\": \"https://****.s3.amazonaws.com/\", 
   \"fields\": 
        {\"key\": \"******\", \"x-amz-algorithm\": \"*******\", \"x-amz-credential\": \"*******\", \"x-amz-date\": \"*********\", \"x-amz-security-token\": \"********", \"policy\": \"**********\", \"x-amz-signature\": \"*******\"}}

如您所见，在使用生成的预签名 URL 上传文件时，我没有提供 AWSAccessKey 或任何凭据，这是合乎逻辑的，因为预签名 URL 是为使用此类 URL.

时无需提供凭据的外部用户创建的

然而，当尝试运行由 Python 的 Requests 库进行的相同调用时，使用 cURL，请求失败并出现错误：

< HTTP/1.1 403 Forbidden
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><Error>

要获得 requests.post 发出的确切请求调用，我运行ning:

req = http_response.request
command = "curl -X {method} -H {headers} -d '{data}' '{uri}'"
method = "PUT"
uri = req.url
data = req.body
headers = ['"{0}: {1}"'.format(k, v) for k, v in req.headers.items()]
headers = " -H ".join(headers)
print(command.format(method=method, headers=headers, data=data, uri=uri))

哪个returns:

curl -v -X PUT -H "Connection: keep-alive" --upload-file xxxx.zip -H "Accept-Encoding: gzip, deflate" -H "Accept: */*" -H "User-Agent: python-requests/2.18.4" -H "Content-Length: xxxx" -H "Content-Type: multipart/form-data; boundary=8a9864bdxxxxx00100ba04cc055a" -d '--8a9864bd377041xxxxx04cc055a
Content-Disposition: form-data; name="x-amz-algorithm"
AWS4-HMAC-SHA256

--8a9864bd377041e0b00100ba04cc055a
Content-Disposition: form-data; name="key"
xxxxx.zip

--8a9864bd377041e0b00100ba04cc055a
Content-Disposition: form-data; name="x-amz-signature"
*****

--8a9864bd377041e0b00100ba04cc055a
Content-Disposition: form-data; name="x-amz-security-token"
*****

--8a9864bd377041e0b00100ba04cc055a
Content-Disposition: form-data; name="x-amz-date"
*****

--8a9864bd377041e0b00100ba04cc055a
Content-Disposition: form-data; name="policy"
*****

--8a9864bd377041e0b00100ba04cc055a
Content-Disposition: form-data; name="x-amz-credential"
xxxxx/xxxxx/xxxx/s3/aws4_request

' 'https://xxxxx.s3.amazonaws.com/'

然后重新制定：

$ curl -v -T file "https://****.s3.amazonaws.com/?key=************&x-amz-algorithm=***************&x-amz-credential=*************&x-amz-security-token=************&policy=**********&x-amz-signature=****************

经过研究，我没有发现与此问题类似的问题，但是： https://aws.amazon.com/es/premiumsupport/knowledge-center/s3-access-denied-error/

这对我来说仍然不合逻辑，因为在使用预签名 URL 时我不应该输入任何凭据。

我不知道我是否遗漏了 Python 的 Requests 图书馆提出的完整请求。

任何想法，请！

亲切的问候，

Rshad

Answer 1

这个简单的 curl 命令应该可以工作：

使用通常的预签名 url，它将如下所示：

curl -v \
-F key=<filename> \
-F x-amz-algorithm=*** \
-F x-amz-credential=*** \
-F x-amz-date=*** \
-F x-amz-security-token=*** \
-F policy=*** \
-F x-amz-signature=*** \
-F file=@<filename> \
'https://<bucket>.s3.amazonaws.com/'

-F 字段允许您指定应上传到 S3 的额外 POST 数据（即从返回的带有预签名 URL 的字段数据。

亲切的问候，

AWS Presigned URL 与 Python 的 Requests 库一起工作，但在 cURL 下失败

AWS Presigned URL works with Python's Requests library but fails with cURL

curl

amazon-web-services

python-requests

pre-signed-url