从 google 存储中复制和删除文件，但收到 python 错误作为 quote_from_bytes() 预期字节

Question

我正在尝试实现一项要求，我们应该在流程结束后将文件从 google 云存储中的一个特定目录移动到另一个存档目录，以便第二天前一天的文件未处理。

我看了下面 post 来实现一个解决方案 How to move files in Google Cloud Storage from one bucket to another bucket by Python

我在云函数下面python写了

#!/usr/bin/python
# -*- coding: utf-8 -*-
from google.cloud import storage


def mv_blob(bucket_name,source_blobs):
    """Copies a blob from one bucket to another with a new name."""

    bucket_name = 'whr-asia-datalake-dev-standard'
    directory_name = 'outbound/Archive/'
    destination_bucket_name = 'whr-asia-datalake-dev-standard'

    storage_client = storage.Client()

    source_bucket = storage_client.bucket(bucket_name)
    source_blobs = source_bucket.list_blobs(prefix=directory_name)
    destination_bucket = storage_client.bucket(destination_bucket_name)

    for blob in source_blobs:
        blob_copy = source_bucket.copy_blob(
        blob, destination_bucket, blob
        )
        blob.delete()
        print(
        "Blob {} in bucket {} copied to blob {} in bucket {}.".format(
            source_blob.name,
            source_bucket.name,
            blob_copy.name,
            destination_bucket.name,

) )

但是，我在 python 中收到“quote_from_bytes() 预期字节”错误，请查看执行详情

File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 449, in run_background_function
    _function_handler.invoke_user_function(event_object)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 268, in invoke_user_function
    return call_user_function(request_or_event)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 265, in call_user_function
    event_context.Context(**request_or_event.context))
  File "/user_code/main.py", line 21, in mv_blob
    blob, destination_bucket, blob
  File "/env/local/lib/python3.7/site-packages/google/cloud/storage/bucket.py", line 487, in copy_blob
    api_path = blob.path + '/copyTo' + new_blob.path
  File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 171, in path
    return self.path_helper(self.bucket.path, self.name)
  File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 146, in path_helper
    return bucket_path + '/o/' + quote(blob_name, safe='')
  File "/opt/python3.7/lib/python3.7/urllib/parse.py", line 834, in quote
    return quote_from_bytes(string, safe)
  File "/opt/python3.7/lib/python3.7/urllib/parse.py", line 859, in quote_from_bytes
    raise TypeError("quote_from_bytes() expected bytes")
TypeError: quote_from_bytes() expected bytes

删除部分工作正常，但只要我在我的函数中添加下面，它就会出现这个问题

blob_copy = source_bucket.copy_blob(
        blob, destination_bucket, blob
        )

有人可以帮忙解决这个问题吗？感谢您的意见和帮助，

Answer 1

试着打印blob，你就会很快明白！事实上，在云存储中，没有目录。你只有斑点；以及完整路径中的名称 (path/to/file).

在此背景下，动脑编代码

blob_copy = source_bucket.copy_blob(
        blob, destination_bucket, blob
        )

从源存储桶 whr-asia-datalake-dev-standard 将 blob outbound/Archive/XXX 复制到存储桶目标 whr-asia-datalake-dev-standard 并使用 blob 名称 outbound/Archive/XXX

你应该感觉到了问题：来源和目的地是一样的！

从 google 存储中复制和删除文件，但收到 python 错误作为 quote_from_bytes() 预期字节

Copying and deleting files from google storage but getting python error as quote_from_bytes() expected bytes

python

google-cloud-functions

google-cloud-storage

delete-file