client.get_bucket() 失败，但仅来自云数据流（计算引擎）

Question

应用程序一直正常运行，现在正在重新部署，google存储出现奇怪的错误。

MissingSchema：无效 URL 'None/storage/v1/b/my-bucket-name?projection=noAcl'：未提供架构。也许你的意思是 http://None/storage/v1/b/my-bucket-name?projection=noAcl?

  File "/usr/local/lib/python2.7/dist-packages/lib/file_store.py", line 11, in __init__
    self.bucket = self.client.get_bucket(parts[0])
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/storage/client.py", line 301, in get_bucket
    bucket.reload(client=self)
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/storage/_helpers.py", line 130, in reload
    _target_object=self,
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/_http.py", line 392, in api_request
    target_object=_target_object,
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/_http.py", line 269, in _make_request
    return self._do_request(method, url, headers, data, target_object)
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/_http.py", line 298, in _do_request
    return self.http.request(url=url, method=method, headers=headers, data=data)
  File "/usr/local/lib/python2.7/dist-packages/google/auth/transport/requests.py", line 208, in request
    method, url, data=data, headers=request_headers, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 519, in request
    prep = self.prepare_request(req)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 462, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 313, in prepare
    self.prepare_url(url, params)
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 387, in prepare_url
    raise MissingSchema(error)
MissingSchema: Invalid URL 'None/storage/v1/b/my-bucket-name?projection=noAcl': No schema supplied. Perhaps you meant http://None/storage/v1/b/my-bucket-name?projection=noAcl? [while running 'generatedPtransform-51']

导致错误的代码，我可以运行使用相同的服务帐户在本地进行此操作，并且可以正常工作，没有错误。我正在使用 $env:GOOGLE_APPLICATION_CREDENTIALS 在部署时导出我的服务帐户凭据。所有其他服务都正常工作。

# My test is:
# fs = FileStore("gs://my-bucket-name/models/", "development", "general")

class FileStore():
    # modelPath - must be a gs:// style google storage resource path containing everything but the file extension
    def __init__(self, modelPath, env, modelName):
        from google.cloud import storage

        parts = modelPath[5:].split('/', 1)
        self.client = storage.Client()
        self.bucket = self.client.get_bucket(parts[0]) # <- error here

为什么 google 核心客户端无法构建 URL？根据 'None/storage/v1/b/my-bucket-name?projection=noAcl'，URL 的缺失部分应该类似于“https://www.googleapis.com”。

Answer 1

这个错误显然是由 google_cloud_storage 和 google_cloud_core 之间的不匹配引起的。我在我的 setup.py 中指定了 google_cloud_core >= 1.0.3 但是当我查看计算 VM 上的 docker 图像时我发现它有一个早期版本。

从 setup.py 重建我的 venv 后，我还必须运行:

C:\Python27\python.exe -m pipenv install google-cloud-core>=1.0.3 --skip-lock

然后我就可以部署了，应用程序又开始工作了。

client.get_bucket() 失败，但仅来自云数据流（计算引擎）

client.get_bucket() fails, but only from cloud dataflow (compute engine)

python

google-cloud-storage

google-cloud-dataflow