python AWS boto3 为文件上传创建预签名 url
python AWS boto3 create presigned url for file upload
我正在为客户端将视频文件上传到 s3 的应用程序编写 django 后端。我想使用预先签名的 urls,所以 django 服务器将签署一个 url 并将其传递回客户端,然后客户端将他们的视频上传到 s3。问题是,generate_presigned_url 方法似乎不知道 s3 客户端 upload_file 方法...
在此 example 之后,我使用以下代码生成用于上传的 url:
s3_client = boto3.client('s3')
try:
s3_object_name = str(uuid4()) + file_extension
params = {
"file_name": local_filename,
"bucket": settings.VIDEO_UPLOAD_BUCKET_NAME,
"object_name": s3_object_name,
}
response = s3_client.generate_presigned_url(ClientMethod="upload_file",
Params=params,
ExpiresIn=500)
except ClientError as e:
logging.error(e)
return HttpResponse(503, reason="Could not retrieve upload url.")
当 运行 时,我得到错误:
File "/Users/bridgedudley/.local/share/virtualenvs/ShoMe/lib/python3.6/site-packages/botocore/signers.py", line 574, in generate_presigned_url
operation_name = self._PY_TO_OP_NAME[client_method]
KeyError: 'upload_file'
哪个触发异常:
botocore.exceptions.UnknownClientMethodError: Client does not have method: upload_file
调试后我发现 self._PY_TO_OP_NAME 字典只包含 here 提供的 s3 客户端命令的一个子集:
向下滚动到 "upload"...
没有upload_file方法!我使用 "list_buckets" 尝试了相同的代码,它工作得很好,给了我一个预先签名的 url,其中列出了签名者凭据下的存储桶。
所以 generate_presigned_url 函数中没有可用的 upload_file 方法,我怎样才能实现我想要的功能?
谢谢!
您应该可以在这里使用 put_object
方法。它是一个纯客户端对象,而不是像 upload_file
这样的元客户端对象。这就是 upload_file
没有出现在 client._PY_TO_OP_NAME
中的原因。这两个函数确实接受不同的输入,这可能需要对您的代码进行轻微重构。
put_object: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.put_object
除了已经提到的用法:
boto3.client('s3').generate_presigned_url('put_object', Params={'Bucket':'your-bucket-name', 'Key':'your-object-name'})
您还可以使用:
boto3.client('s3').generate_presigned_post('your-bucket_name', 'your-object_name')
URL 的样本生成:
import boto3
bucket_name = 'my-bucket'
key_name = 'any-name.txt'
s3_client = boto3.client('s3')
upload_details = s3_client.generate_presigned_post(bucket_name, key_name)
print(upload_details)
输出:
{'url': 'https://my-bucket.s3.amazonaws.com/', 'fields': {'key': 'any-name.txt', 'AWSAccessKeyId': 'QWERTYUOP123', 'x-amz-security-token': 'a1s2d3f4g5h6j7k8l9', 'policy': 'z0x9c8v7b6n5m4', 'signature': 'qaz123wsx456edc'}}
文件上传示例:
import requests
filename_to_upload = './some-file.txt'
with open(filename_to_upload, 'rb') as file_to_upload:
files = {'file': (filename_to_upload, file_to_upload)}
upload_response = requests.post(upload_details['url'], data=upload_details['fields'], files=files)
print(f"Upload response: {upload_response.status_code}")
输出:
Upload response: 204
补充说明:
如文件所示:
The credentials used by the presigned URL are those of the AWS user
who generated the URL.
因此,确保将执行这一代预签名 URL 的实体允许策略 s3:PutObject
能够使用签名 URL 将文件上传到 S3 .创建后,可以通过不同的方式对其进行配置。其中一些是:
作为 Lambda 函数的允许策略
或通过boto3:
s3_client = boto3.client('s3',
aws_access_key_id="your-access-key-id",
aws_secret_access_key="your-secret-access-key",
aws_session_token="your-session-token", # Only for credentials that has it
)
或者关于工作环境:
# Run in the Linux environment
export AWS_ACCESS_KEY_ID="your-access-key-id"
export AWS_SECRET_ACCESS_KEY="your-secret-access-key"
export AWS_SESSION_TOKEN="your-session-token", # Only for credentials that has it
或通过图书馆,例如Django 的 django 存储
我正在为客户端将视频文件上传到 s3 的应用程序编写 django 后端。我想使用预先签名的 urls,所以 django 服务器将签署一个 url 并将其传递回客户端,然后客户端将他们的视频上传到 s3。问题是,generate_presigned_url 方法似乎不知道 s3 客户端 upload_file 方法...
在此 example 之后,我使用以下代码生成用于上传的 url:
s3_client = boto3.client('s3')
try:
s3_object_name = str(uuid4()) + file_extension
params = {
"file_name": local_filename,
"bucket": settings.VIDEO_UPLOAD_BUCKET_NAME,
"object_name": s3_object_name,
}
response = s3_client.generate_presigned_url(ClientMethod="upload_file",
Params=params,
ExpiresIn=500)
except ClientError as e:
logging.error(e)
return HttpResponse(503, reason="Could not retrieve upload url.")
当 运行 时,我得到错误:
File "/Users/bridgedudley/.local/share/virtualenvs/ShoMe/lib/python3.6/site-packages/botocore/signers.py", line 574, in generate_presigned_url operation_name = self._PY_TO_OP_NAME[client_method] KeyError: 'upload_file'
哪个触发异常:
botocore.exceptions.UnknownClientMethodError: Client does not have method: upload_file
调试后我发现 self._PY_TO_OP_NAME 字典只包含 here 提供的 s3 客户端命令的一个子集:
向下滚动到 "upload"...
没有upload_file方法!我使用 "list_buckets" 尝试了相同的代码,它工作得很好,给了我一个预先签名的 url,其中列出了签名者凭据下的存储桶。
所以 generate_presigned_url 函数中没有可用的 upload_file 方法,我怎样才能实现我想要的功能?
谢谢!
您应该可以在这里使用 put_object
方法。它是一个纯客户端对象,而不是像 upload_file
这样的元客户端对象。这就是 upload_file
没有出现在 client._PY_TO_OP_NAME
中的原因。这两个函数确实接受不同的输入,这可能需要对您的代码进行轻微重构。
put_object: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.put_object
除了已经提到的用法:
boto3.client('s3').generate_presigned_url('put_object', Params={'Bucket':'your-bucket-name', 'Key':'your-object-name'})
您还可以使用:
boto3.client('s3').generate_presigned_post('your-bucket_name', 'your-object_name')
URL 的样本生成:
import boto3
bucket_name = 'my-bucket'
key_name = 'any-name.txt'
s3_client = boto3.client('s3')
upload_details = s3_client.generate_presigned_post(bucket_name, key_name)
print(upload_details)
输出:
{'url': 'https://my-bucket.s3.amazonaws.com/', 'fields': {'key': 'any-name.txt', 'AWSAccessKeyId': 'QWERTYUOP123', 'x-amz-security-token': 'a1s2d3f4g5h6j7k8l9', 'policy': 'z0x9c8v7b6n5m4', 'signature': 'qaz123wsx456edc'}}
文件上传示例:
import requests
filename_to_upload = './some-file.txt'
with open(filename_to_upload, 'rb') as file_to_upload:
files = {'file': (filename_to_upload, file_to_upload)}
upload_response = requests.post(upload_details['url'], data=upload_details['fields'], files=files)
print(f"Upload response: {upload_response.status_code}")
输出:
Upload response: 204
补充说明:
如文件所示:
The credentials used by the presigned URL are those of the AWS user who generated the URL.
因此,确保将执行这一代预签名 URL 的实体允许策略 s3:PutObject
能够使用签名 URL 将文件上传到 S3 .创建后,可以通过不同的方式对其进行配置。其中一些是:
作为 Lambda 函数的允许策略
或通过boto3:
s3_client = boto3.client('s3',
aws_access_key_id="your-access-key-id",
aws_secret_access_key="your-secret-access-key",
aws_session_token="your-session-token", # Only for credentials that has it
)
或者关于工作环境:
# Run in the Linux environment
export AWS_ACCESS_KEY_ID="your-access-key-id"
export AWS_SECRET_ACCESS_KEY="your-secret-access-key"
export AWS_SESSION_TOKEN="your-session-token", # Only for credentials that has it
或通过图书馆,例如Django 的 django 存储