如何在 Google Cloud 运行中生成签名 url 的 Blob？

Question

在 Google 云运行下，您可以 select 您的容器是哪个服务帐户运行。使用默认计算服务帐户无法生成已签名的 url.

此处列出的解决方法适用于 Google 云计算——如果您允许服务帐户的所有范围。在 Cloud 运行中似乎没有办法做到这一点（我找不到）。

https://github.com/googleapis/google-auth-library-python/issues/50

我尝试过的事情：

为服务帐户分配了角色：roles/iam.serviceAccountTokenCreator
在虚拟机中的同一个 GCP 项目中验证了解决方法（对比云运行）
验证代码在容器中本地工作，服务帐户从私钥加载（通过 json 文件）。

from google.cloud import storage
client = storage.Client()
bucket = client.get_bucket('EXAMPLE_BUCKET')
blob = bucket.get_blob('libraries/image_1.png')
expires = datetime.now() + timedelta(seconds=86400)
blob.generate_signed_url(expiration=expires)

失败：

you need a private key to sign credentials.the credentials you are currently using <class 'google.auth.compute_engine.credentials.Credentials'> just contains a token. see https://googleapis.dev/python/google-api-core/latest/auth.html#setting-up-a-service-account for more details.
/usr/local/lib/python3.8/site-packages/google/cloud/storage/_signing.py, line 51, in ensure_signed_credentials

正在尝试添加解决方法，

Error calling the IAM signBytes API: 
{  "error": {  "code": 400,

    "message": "Request contains an invalid argument.",
    "status": "INVALID_ARGUMENT"  }
}
Exception Location: /usr/local/lib/python3.8/site-packages/google/auth/iam.py, line 81, in _make_signing_request

Github 问题中提到的解决方法代码：

from google.cloud import storage
from google.auth.transport import requests
from google.auth import compute_engine
from datetime import datetime, timedelta

def get_signing_creds(credentials):
    auth_request = requests.Request()
    print(credentials.service_account_email)
    signing_credentials = compute_engine.IDTokenCredentials(auth_request, "", service_account_email=credentials.ser
vice_account_email)
    return signing_credentials


client = storage.Client()
bucket = client.get_bucket('EXAMPLE_BUCKET')
blob = bucket.get_blob('libraries/image_1.png')
expires = datetime.now() + timedelta(seconds=86400)
signing_creds = get_signing_creds(client._credentials)
url = blob.generate_signed_url(expiration=expires, credentials=signing_creds)
print(url)

如何在 Google Cloud 运行下生成签名的 url？在这一点上，我似乎必须安装我想避免的服务帐户密钥。

编辑：尝试澄清一下，服务帐户具有正确的权限 - 它在 GCE 和本地使用 JSON 私钥工作。

Answer 1

您无法使用默认服务帐户对网址进行签名。

使用具有权限的专用服务帐户再次尝试您的服务代码，看看是否可以解决您的错误

参考资料和进一步阅读：

Answer 2

是的，你可以，但我必须深入研究才能找到方法（如果你不关心细节，请跳到最后）

如果你进去_signing.py file, line 623，你可以看到这个

if access_token and service_account_email:
   signature = _sign_message(string_to_sign, access_token, service_account_email)
...

如果你提供access_token和service_account_email，你可以使用_sign_message方法。此方法使用 IAM service SignBlob API at this line

这很重要，因为您现在可以在没有本地私钥的情况下对 blob 进行签名！！所以，这解决了问题，下面的代码在 Cloud 运行上工作（我确定在 Cloud Function 上工作）

def sign_url():
    from google.cloud import storage
    from datetime import datetime, timedelta

    import google.auth
    credentials, project_id = google.auth.default()

    # Perform a refresh request to get the access token of the current credentials (Else, it's None)
    from google.auth.transport import requests
    r = requests.Request()
    credentials.refresh(r)

    client = storage.Client()
    bucket = client.get_bucket('EXAMPLE_BUCKET')
    blob = bucket.get_blob('libraries/image_1.png')
    expires = datetime.now() + timedelta(seconds=86400)

    # In case of user credential use, define manually the service account to use (for development purpose only)
    service_account_email = "YOUR DEV SERVICE ACCOUNT"
    # If you use a service account credential, you can use the embedded email
    if hasattr(credentials, "service_account_email"):
        service_account_email = credentials.service_account_email

    url = blob.generate_signed_url(expiration=expires,service_account_email=service_account_email, access_token=credentials.token)
    return url, 200

如果不清楚请告诉我

Answer 3

已在 GCP documentation 中为 Cloud 运行和 App Engine 等无服务器实例添加了更新方法。

下面的snippet展示了如何从存储库创建签名URL。

def generate_upload_signed_url_v4(bucket_name, blob_name):
    """Generates a v4 signed URL for uploading a blob using HTTP PUT.

    Note that this method requires a service account key file. You can not use
    this if you are using Application Default Credentials from Google Compute
    Engine or from the Google Cloud SDK.
    """
    # bucket_name = 'your-bucket-name'
    # blob_name = 'your-object-name'

    storage_client = storage.Client()
    bucket = storage_client.bucket(bucket_name)
    blob = bucket.blob(blob_name)

    url = blob.generate_signed_url(
        version="v4",
        # This URL is valid for 15 minutes
        expiration=datetime.timedelta(minutes=15),
        # Allow PUT requests using this URL.
        method="PUT",
        content_type="application/octet-stream",
    )


    return url

一旦你的后端 returns 签名 URL 你可以从你的前端执行 curl put 请求，如下所示

curl -X PUT -H 'Content-Type: application/octet-stream' --upload-file my-file 'my-signed-url'

Answer 4

答案 @guillaume-blaquiere posted 确实有效，但它需要一个未提及的额外步骤，即将 IAM 中的 Service Account Token Creator 角色添加到您的 default 服务帐户，这将允许所述默认服务帐户“模拟服务帐户（创建 OAuth2 访问令牌、签署 blob 或 JWT 等）”。

这允许 默认服务帐户 根据 signBlob documentation.

对 blob 进行签名

我在 AppEngine 上试过了，一旦获得许可，它就可以完美运行。

import datetime as dt

from google import auth
from google.cloud import storage

# SCOPES = [
#     "https://www.googleapis.com/auth/devstorage.read_only",
#     "https://www.googleapis.com/auth/iam"
# ]

credentials, project = auth.default(
#     scopes=SCOPES
)
credentials.refresh(auth.transport.requests.Request())

expiration_timedelta = dt.timedelta(days=1)

storage_client = storage.Client(credentials=credentials)
bucket = storage_client.get_bucket("bucket_name")
blob = bucket.get_blob("blob_name")

signed_url = blob.generate_signed_url(
    expiration=expiration_timedelta,
    service_account_email=credentials.service_account_email,
    access_token=credentials.token,
)

我下载了 AppEngine 默认服务帐户 的密钥以在本地进行测试，为了使其在 AppEngine 环境之外正常工作，我必须添加适当的范围根据设置 SCOPES 的注释行添加到凭据。如果运行仅在 AppEngine 本身中，您可以忽略它们。

如何在 Google Cloud 运行中生成签名 url 的 Blob？

How to generate a Blob signed url in Google Cloud Run?

python

google-cloud-storage

google-iam

google-cloud-run

如何在 Google Cloud 运行 中生成签名 url 的 Blob？

How to generate a Blob signed url in Google Cloud Run?

python

google-cloud-storage

google-iam

google-cloud-run

如何在 Google Cloud 运行中生成签名 url 的 Blob？