gsutil rsync 是否可以仅允许上传文件的应用程序? (ResumableUploadAbortException:没有 storage.objects.delete 访问权限)

Is gsutil rsync able for app with permission only for upload files? (ResumableUploadAbortException: does not have storage.objects.delete access)

我使用了 gsutil rsync 没有 -d 参数,即使没有任何参数,但我仍然得到 ResumableUploadAbortException: 403 abc@xyz.iam.gserviceaccount.com does not have storage.objects.delete access.

我想创建一个仅用于将文件上传到 google 云存储的应用程序,使用来自 google 计算引擎的 gsutil rsync

我已添加阅读和列表权限, 因为我不希望应用程序能够删除存储桶中的任何文件,所以我没有授予 storage.objects.delete.

权限

是否有任何参数可以使用 gsutil rsync 命令而无需授予删除文件的权限? 我想使用 rsync 因为我必须传输千兆字节的文件。

谢谢。

[编辑] 添加附加信息并调试:

我想添加更多信息, 此应用程序 运行作为 cron,上传时间戳生成的文件名,因此每次执行时都会不同。

上传成功了,文件上传到了bucket,但是最后还是得到了ResumableUploadAbortException

所以我运行gsutil -D rsync调试:

DEBUG: Exception stack trace:
    Traceback (most recent call last):
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 590, in _RunNamedCommandAndHandleExceptions
        user_project=user_project)
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 372, in RunNamedCommand
        return_code = command_inst.RunCommand()
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1546, in RunCommand
        fail_on_error=True, seek_ahead_iterator=seek_ahead_iterator)
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1420, in Apply
        arg_checker, should_return_results, fail_on_error)
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1491, in _SequentialApply
        worker_thread.PerformTask(task, self)
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/command.py", line 2157, in PerformTask
        results = task.func(cls, task.args, thread_state=self.thread_gsutil_api)
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1337, in _RsyncFunc
        gzip_exts=cls.gzip_exts, preserve_posix=cls.preserve_posix_attrs)
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 3520, in PerformCopy
        allow_splitting=allow_splitting, gzip_encoded=gzip_encoded)
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2026, in _UploadFileToObject
        parallel_composite_upload, logger)
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 1877, in _DelegateUploadFileToObject
        elapsed_time, uploaded_object = upload_delegate()
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2015, in CallResumableUpload
        is_component=is_component, gzip_encoded=gzip_encoded_file)
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 1687, in _UploadFileToObjectResumable
        progress_callback=progress_callback, gzip_encoded=gzip_encoded)
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 315, in UploadObjectResumable
        gzip_encoded=gzip_encoded)
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1415, in UploadObjectResumable
        gzip_encoded=gzip_encoded)
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1242, in _UploadObject
        additional_headers, progress_callback, gzip_encoded)
      File "/usr/lib64/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1371, in _PerformResumableUpload
        raise resumable_ex
    ResumableUploadAbortException: ResumableUploadAbortException: 403 abc@xyz.iam.gserviceaccount.com does not have storage.objects.delete access to bucket/filename

如果我不得不猜测,您正在尝试覆盖现有对象。这将删除现有对象,将其替换为另一个同名对象。

一种替代方法可能是gsutil cp -n,它不会覆盖目的地的现有文件。但是,如果您希望能够覆盖对象,则需要删除权限。

编辑:

如以下评论中所述,这也可能是由于启用了并行复合上传(在 gsutil 尝试删除组件对象时)。可以通过在 boto 文件中设置 parallel_composite_upload_threshold=0 或在命令中内联来禁用此行为,例如gsutil -o "GSUtil:parallel_composite_upload_threshold=0" ....