gsutil rsync 是否可以仅允许上传文件的应用程序? (ResumableUploadAbortException:没有 storage.objects.delete 访问权限)
Is gsutil rsync able for app with permission only for upload files? (ResumableUploadAbortException: does not have storage.objects.delete access)
我使用了 gsutil rsync
没有 -d
参数,即使没有任何参数,但我仍然得到 ResumableUploadAbortException: 403 abc@xyz.iam.gserviceaccount.com does not have storage.objects.delete access
.
我想创建一个仅用于将文件上传到 google 云存储的应用程序,使用来自 google 计算引擎的 gsutil rsync
。
我已添加阅读和列表权限,
因为我不希望应用程序能够删除存储桶中的任何文件,所以我没有授予 storage.objects.delete
.
权限
是否有任何参数可以使用 gsutil rsync
命令而无需授予删除文件的权限?
我想使用 rsync
因为我必须传输千兆字节的文件。
谢谢。
[编辑] 添加附加信息并调试:
我想添加更多信息,
此应用程序 运行作为 cron,上传时间戳生成的文件名,因此每次执行时都会不同。
上传成功了,文件上传到了bucket,但是最后还是得到了ResumableUploadAbortException
所以我运行gsutil -D rsync
调试:
DEBUG: Exception stack trace:
Traceback (most recent call last):
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 590, in _RunNamedCommandAndHandleExceptions
user_project=user_project)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 372, in RunNamedCommand
return_code = command_inst.RunCommand()
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1546, in RunCommand
fail_on_error=True, seek_ahead_iterator=seek_ahead_iterator)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1420, in Apply
arg_checker, should_return_results, fail_on_error)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1491, in _SequentialApply
worker_thread.PerformTask(task, self)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/command.py", line 2157, in PerformTask
results = task.func(cls, task.args, thread_state=self.thread_gsutil_api)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1337, in _RsyncFunc
gzip_exts=cls.gzip_exts, preserve_posix=cls.preserve_posix_attrs)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 3520, in PerformCopy
allow_splitting=allow_splitting, gzip_encoded=gzip_encoded)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2026, in _UploadFileToObject
parallel_composite_upload, logger)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 1877, in _DelegateUploadFileToObject
elapsed_time, uploaded_object = upload_delegate()
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2015, in CallResumableUpload
is_component=is_component, gzip_encoded=gzip_encoded_file)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 1687, in _UploadFileToObjectResumable
progress_callback=progress_callback, gzip_encoded=gzip_encoded)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 315, in UploadObjectResumable
gzip_encoded=gzip_encoded)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1415, in UploadObjectResumable
gzip_encoded=gzip_encoded)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1242, in _UploadObject
additional_headers, progress_callback, gzip_encoded)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1371, in _PerformResumableUpload
raise resumable_ex
ResumableUploadAbortException: ResumableUploadAbortException: 403 abc@xyz.iam.gserviceaccount.com does not have storage.objects.delete access to bucket/filename
如果我不得不猜测,您正在尝试覆盖现有对象。这将删除现有对象,将其替换为另一个同名对象。
一种替代方法可能是gsutil cp -n
,它不会覆盖目的地的现有文件。但是,如果您希望能够覆盖对象,则需要删除权限。
编辑:
如以下评论中所述,这也可能是由于启用了并行复合上传(在 gsutil 尝试删除组件对象时)。可以通过在 boto 文件中设置 parallel_composite_upload_threshold=0
或在命令中内联来禁用此行为,例如gsutil -o "GSUtil:parallel_composite_upload_threshold=0" ...
.
我使用了 gsutil rsync
没有 -d
参数,即使没有任何参数,但我仍然得到 ResumableUploadAbortException: 403 abc@xyz.iam.gserviceaccount.com does not have storage.objects.delete access
.
我想创建一个仅用于将文件上传到 google 云存储的应用程序,使用来自 google 计算引擎的 gsutil rsync
。
我已添加阅读和列表权限,
因为我不希望应用程序能够删除存储桶中的任何文件,所以我没有授予 storage.objects.delete
.
是否有任何参数可以使用 gsutil rsync
命令而无需授予删除文件的权限?
我想使用 rsync
因为我必须传输千兆字节的文件。
谢谢。
[编辑] 添加附加信息并调试:
我想添加更多信息, 此应用程序 运行作为 cron,上传时间戳生成的文件名,因此每次执行时都会不同。
上传成功了,文件上传到了bucket,但是最后还是得到了ResumableUploadAbortException
所以我运行gsutil -D rsync
调试:
DEBUG: Exception stack trace:
Traceback (most recent call last):
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 590, in _RunNamedCommandAndHandleExceptions
user_project=user_project)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 372, in RunNamedCommand
return_code = command_inst.RunCommand()
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1546, in RunCommand
fail_on_error=True, seek_ahead_iterator=seek_ahead_iterator)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1420, in Apply
arg_checker, should_return_results, fail_on_error)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1491, in _SequentialApply
worker_thread.PerformTask(task, self)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/command.py", line 2157, in PerformTask
results = task.func(cls, task.args, thread_state=self.thread_gsutil_api)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1337, in _RsyncFunc
gzip_exts=cls.gzip_exts, preserve_posix=cls.preserve_posix_attrs)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 3520, in PerformCopy
allow_splitting=allow_splitting, gzip_encoded=gzip_encoded)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2026, in _UploadFileToObject
parallel_composite_upload, logger)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 1877, in _DelegateUploadFileToObject
elapsed_time, uploaded_object = upload_delegate()
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2015, in CallResumableUpload
is_component=is_component, gzip_encoded=gzip_encoded_file)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 1687, in _UploadFileToObjectResumable
progress_callback=progress_callback, gzip_encoded=gzip_encoded)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 315, in UploadObjectResumable
gzip_encoded=gzip_encoded)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1415, in UploadObjectResumable
gzip_encoded=gzip_encoded)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1242, in _UploadObject
additional_headers, progress_callback, gzip_encoded)
File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1371, in _PerformResumableUpload
raise resumable_ex
ResumableUploadAbortException: ResumableUploadAbortException: 403 abc@xyz.iam.gserviceaccount.com does not have storage.objects.delete access to bucket/filename
如果我不得不猜测,您正在尝试覆盖现有对象。这将删除现有对象,将其替换为另一个同名对象。
一种替代方法可能是gsutil cp -n
,它不会覆盖目的地的现有文件。但是,如果您希望能够覆盖对象,则需要删除权限。
编辑:
如以下评论中所述,这也可能是由于启用了并行复合上传(在 gsutil 尝试删除组件对象时)。可以通过在 boto 文件中设置 parallel_composite_upload_threshold=0
或在命令中内联来禁用此行为,例如gsutil -o "GSUtil:parallel_composite_upload_threshold=0" ...
.