配置 Concourse CI 以使用 AWS Secrets Manager

Question

我一直在想办法配置 docker 版本的 Concourse (https://github.com/concourse/concourse-docker) to use the AWS Secrets Manager and I added the following environment variables into the docker-compose file but from the logs it doesn't look like it ever reaches out to AWS to fetch the creds. Am I missing something or should this automatically happen when adding these environment variables under environment in the docker-compose file? Here are the docs I have been looking at https://concourse-ci.org/aws-asm-credential-manager.html

version: '3'

services:
  concourse-db:
    image: postgres
    environment:
      POSTGRES_DB: concourse
      POSTGRES_PASSWORD: concourse_pass
      POSTGRES_USER: concourse_user
      PGDATA: /database

  concourse:
    image: concourse/concourse
    command: quickstart
    privileged: true
    depends_on: [concourse-db]
    ports: ["9090:8080"]
    environment:
      CONCOURSE_POSTGRES_HOST: concourse-db
      CONCOURSE_POSTGRES_USER: concourse_user
      CONCOURSE_POSTGRES_PASSWORD: concourse_pass
      CONCOURSE_POSTGRES_DATABASE: concourse
      CONCOURSE_EXTERNAL_URL: http://XXX.XXX.XXX.XXX:9090
      CONCOURSE_ADD_LOCAL_USER: test: test
      CONCOURSE_MAIN_TEAM_LOCAL_USER: test
      CONCOURSE_WORKER_BAGGAGECLAIM_DRIVER: overlay
      CONCOURSE_AWS_SECRETSMANAGER_REGION: us-east-1
      CONCOURSE_AWS_SECRETSMANAGER_ACCESS_KEY: <XXXX>
      CONCOURSE_AWS_SECRETSMANAGER_SECRET_KEY: <XXXX>
      CONCOURSE_AWS_SECRETSMANAGER_TEAM_SECRET_TEMPLATE: /concourse/{{.Secret}}
      CONCOURSE_AWS_SECRETSMANAGER_PIPELINE_SECRET_TEMPLATE: /concourse/{{.Secret}}

pipeline.yml 示例：

jobs:
  - name: build-ui
    plan:
      - get: web-ui
        trigger: true
      - get: resource-ui
      - task: build-task
        file: web-ui/ci/build/task.yml
      - put: resource-ui
        params:
          repository: updated-ui
          force: true
      - task: e2e-task
        file: web-ui/ci/e2e/task.yml
        params:
          UI_USERNAME: ((ui-username))
          UI_PASSWORD: ((ui-password))

resources:
  - name: cf
    type: cf-cli-resource
    source:
      api: https://api.run.pivotal.io
      username: ((cf-username))
      password: ((cf-password))
      org: Blah

  - name: web-ui
    type: git
    source:
      uri: git@github.com:blah/blah.git
      branch: master
      private_key: ((git-private-key))

Answer 1

在 AWS Secrets Manager 中存储 concourse 管道的参数时，必须遵循此语法，

/concourse/TEAM_NAME/PIPELINE_NAME/PARAMETER_NAME`

如果您有跨团队在多个管道中使用的通用参数，请使用此语法避免在机密管理器中创建冗余参数

/concourse/TEAM_NAME/PARAMETER_NAME

支持的最高级别是大厅团队级别。

全局参数是不可能的。因此，您的撰写环境中的这些变量将不受支持。

CONCOURSE_AWS_SECRETSMANAGER_TEAM_SECRET_TEMPLATE: /concourse/{{.Secret}}
CONCOURSE_AWS_SECRETSMANAGER_PIPELINE_SECRET_TEMPLATE: /concourse/{{.Secret}}

除非您想更改前缀/concourse，否则这些参数应保留其默认值。

并且，在管道中检索这些参数时，无需在模板中进行任何更改。只需传递 PARAMETER_NAME，concourse 将根据团队和管道名称处理 secrets manager 中的查找。

...
        params:
          UI_USERNAME: ((ui-username))
          UI_PASSWORD: ((ui-password))
...

配置 Concourse CI 以使用 AWS Secrets Manager

Configuring Concourse CI to use AWS Secrets Manager

amazon-web-services

docker

concourse

concourse-pipeline