GCP：无法通过 gcloud/API 创建项目接收器

Question

我目前有一个场景，其中我使用 Deployment Manager 创建了部署脚本 (python)，它运行在我倾向于尝试使用的 GCP 沙箱 (LA Playground) 中非常好事情出来了。但是，当我尝试运行在具有计费帐户等的实际 GCP 项目上使用它时，即使我在为它创建的服务帐户中使用基本相同的角色集，我也会收到权限错误。我也是那个麻烦的 GCP 帐户的项目所有者。

我遇到问题的特定权限集是创建项目接收器。我总是收到以下错误：

Error in Operation [operation-xxxxxxxxxxxx-xxxxxxxxxxxxx-xxxxxxxx-9d487530]: errors:
- code: RESOURCE_ERROR
  location: /deployments/structured-pipeline/resources/dataprep-bq-listener-sink
  message: '{"ResourceType":"gcp-types/logging-v2:projects.sinks","ResourceErrorCode":"403","ResourceErrorMessage":{"code":403,"message":"The
    caller does not have permission","status":"PERMISSION_DENIED","statusMessage":"Forbidden","requestPath":"https://logging.googleapis.com/v2/projects/redacted-gcp-project-name-dev/sinks","httpMethod":"POST"}}'

这是我在用于运行部署的服务帐户上附加的角色列表：

计算管理员
部署管理器编辑器
安全管理员
日志管理员
日志配置编写器
Pub/Sub 管理员
服务使用管理员
存储管理员

如您所见，我已经将他们设置为管理员级别，以确保我没有缺少所需的权限。

根据官方文档，创建项目接收器所需的特定权限是 logging.sinks.create，它包含在 Logging Admin 和 Logs Configuration Writer 角色中。

更新#1：

这是问题资源的扩展DM配置

- name: dataprep-bq-listener-sink
  properties:
    destination: pubsub.googleapis.com/projects/redacted-gcp-project-name-dev/topics/dataprep-bq-listener-vpcps3s6wjzbmkbxxlsqqh
    filter: |-
      resource.type="bigquery_dataset"
      resource.labels.dataset_id="dataprep_output_vpcps3s6wjzbmkbxxlsqqh"
      protoPayload.methodName="google.cloud.bigquery.v2.JobService.InsertJob"
    outputVersionFormat: V2
    sink: dataprep-bq-listener-sink-vpcps3s6wjzbmkbxxlsqqh
  type: gcp-types/logging-v2:projects.sinks
- accessControl:
    gcpIamPolicy:
      bindings:
      - members:
        - $(ref.dataprep-bq-listener-sink.writerIdentity)
        role: roles/pubsub.publisher
  name: dataprep-bq-listener
  properties:
    labels:
      client_namespace: redacted-client-name
    topic: dataprep-bq-listener-vpcps3s6wjzbmkbxxlsqqh
  type: pubsub.v1.topic
  metadata:
    dependsOn:
    - dataprep-bq-listener-sink

更新#2

我将 gcloud 设置为使用服务帐户作为凭据 (https://cloud.google.com/sdk/docs/authorizing#authorizing_with_a_service_account)，该帐户具有我上面所述的角色。

运行 gcloud auth list 明明指向服务账号我 created/used.

ACTIVE  ACCOUNT
        my_redacted_email@company.com
        cloud_user_p_2725ef46@linuxacademygclabs.com
        cloud_user_p_2e3db20d@linuxacademygclabs.com
        cloud_user_p_41b5121a@linuxacademygclabs.com
        cloud_user_p_b57aaef8@linuxacademygclabs.com
        cloud_user_p_bdb72060@linuxacademygclabs.com
        cloud_user_p_c2f5d19a@linuxacademygclabs.com
        cloud_user_p_c3c54122@linuxacademygclabs.com
        cloud_user_p_c88350f1@linuxacademygclabs.com
        cloud_user_p_d7702b8b@linuxacademygclabs.com
*       service_account_used_in_dm@project_id.iam.gserviceaccount.com

To set the active account, run:
    $ gcloud config set account `ACCOUNT`

更新#3

运行 gcloud projects get-iam-policy <PROJECT_ID> 结果到以下 IAM 策略列表（我已将我的电子邮件替换为“my_redacted_email@company.com”和我用于 DM 的服务帐户电子邮件“service_account_used_in_dm@project_id.iam.gserviceaccount.com”；所有默认的默认服务帐户我保留了)

bindings:
- members:
  - user: my_redacted_email@company.com
  role: roles/billing.projectManager
- members:
  - serviceAccount:1008104628570@cloudbuild.gserviceaccount.com
  role: roles/cloudbuild.builds.builder
- members:
  - serviceAccount:service-1008104628570@gcp-sa-cloudbuild.iam.gserviceaccount.com
  role: roles/cloudbuild.serviceAgent
- members:
  - serviceAccount:service-1008104628570@gcf-admin-robot.iam.gserviceaccount.com
  role: roles/cloudfunctions.serviceAgent
- members:
  - serviceAccount:service-1008104628570@gcp-sa-cloudscheduler.iam.gserviceaccount.com
  role: roles/cloudscheduler.serviceAgent
- members:
  - serviceAccount:service_account_used_in_dm@project_id.iam.gserviceaccount.com
  role: roles/compute.admin
- members:
  - serviceAccount:service-1008104628570@compute-system.iam.gserviceaccount.com
  role: roles/compute.serviceAgent
- members:
  - serviceAccount:service-1008104628570@containerregistry.iam.gserviceaccount.com
  role: roles/containerregistry.ServiceAgent
- members:
  - serviceAccount:service-1008104628570@dataflow-service-producer-prod.iam.gserviceaccount.com
  role: roles/dataflow.serviceAgent
- members:
  - serviceAccount:service-1008104628570@trifacta-gcloud-prod.iam.gserviceaccount.com
  role: roles/dataprep.serviceAgent
- members:
  - serviceAccount:service_account_used_in_dm@project_id.iam.gserviceaccount.com
  role: roles/deploymentmanager.editor
- members:
  - serviceAccount:1008104628570-compute@developer.gserviceaccount.com
  - serviceAccount:1008104628570@cloudservices.gserviceaccount.com
  - serviceAccount:service_account_used_in_dm@project_id.iam.gserviceaccount.com
  - user:my_redacted_email@company.com
  role: roles/editor
- members:
  - serviceAccount:service_account_used_in_dm@project_id.iam.gserviceaccount.com
  role: roles/iam.securityAdmin
- members:
  - user:my_redacted_email@company.com
  role: roles/iam.serviceAccountUser
- members:
  - serviceAccount:service_account_used_in_dm@project_id.iam.gserviceaccount.com
  role: roles/logging.admin
- members:
  - serviceAccount:service_account_used_in_dm@project_id.iam.gserviceaccount.com
  role: roles/logging.configWriter
- members:
  - user:my_redacted_email@company.com
  role: roles/owner
- members:
  - serviceAccount:service_account_used_in_dm@project_id.iam.gserviceaccount.com
  role: roles/pubsub.admin
- members:
  - user:my_redacted_email@company.com
  role: roles/resourcemanager.projectIamAdmin
- members:
  - serviceAccount:service_account_used_in_dm@project_id.iam.gserviceaccount.com
  role: roles/serviceusage.serviceUsageAdmin
- members:
  - serviceAccount:service_account_used_in_dm@project_id.iam.gserviceaccount.com
  role: roles/storage.admin
etag: BwW0MM3sGXk=
version: 1

您可能已经注意到，我使用的服务帐户绑定了我上面在项目级别提到的角色。

更新#4

使用相同的服务帐户通过 gcloud 创建接收器会成功创建接收器。

gcloud logging sinks create dataprep-bq-listener-sink-vpcps3s6wjzbmkbxxlsqqh pubsub.googleapis.com/projects/<project_id>/topics/dataprep-bq-listener-vpcps3s6wjzbmkbxxlsqqh --log-filter='resource.type="bigquery_dataset" AND resource.labels.dataset_id="dataprep_output_vpcps3s6wjzbmkbxxlsqqh" AND protoPayload.methodName="google.cloud.bigquery.v2.JobService.InsertJob"' --project=<project_id>

更新#5

@Kolban 注意到 Deployment Manager 对 [PROJECT_NUMBER]@cloudservices.gserviceaccount.com 服务帐户对 Editor 角色 per stated in their official documentation 的要求，现在查看 UPDATE#4 ，它清楚地表明该服务帐户具有 Editor 角色。

他还指出可能会混淆多个云帐户（我可能运行使用服务帐户但在不同的云帐户下执行命令），因此，我执行了以下命令：gcloud config get-value account & gcloud config get-value project 结果是我所期望的，这是正确的 GCP 项目和服务帐户对。

Answer 1

docs状态：-

To create other Google Cloud resources, Deployment Manager uses the credentials of the Google APIs service account to authenticate to other APIs. The Google APIs service account is designed specifically to run internal Google processes on your behalf. The service account is identifiable using the email:

[PROJECT_NUMBER]@cloudservices.gserviceaccount.com

在您的情况下，您的 1008104628570@cloudservices.gserviceaccount.com 服务帐户在项目级别绑定到 roles\editor 角色。但是，您需要的权限 (logging.sinks.create) 不包含在该旧编辑者角色中。

您能否尝试另外授予 1008104628570@cloudservices.gserviceaccount.com 服务帐户 Logging Admin 角色 (roles/logging.admin)，看看是否有帮助？

GCP：无法通过 gcloud/API 创建项目接收器

GCP: Unable to create Project Sink through gcloud/API

service-accounts

google-iam

google-cloud-logging

google-deployment-manager