在 GitLab CI 中使用 Paketo.io / CloudNativeBuildpacks (CNB) 与 Kubernetes 执行器和非特权运行器（没有 pack CLI & docker）

Question

我们想用Paketo.io / CloudNativeBuildpacks (CNB) GitLab CI in the most simple way. Our GitLab setup uses an AWS EKS cluster with unprivileged GitLab CI Runners leveraging the Kubernetes executor. We also don't want to introduce security risks by using Docker in our builds。所以我们没有暴露主机的 /var/run/docker.sock 也不想使用 docker:dind.

我们找到了一些关于如何将 Paketo 与 GitLab CI 一起使用的指南 https://tanzu.vmware.com/developer/guides/gitlab-ci-cd-cnb/ 。但正如标题 Use Cloud Native Buildpacks with GitLab in GitLab Build Job WITHOUT Using the GitLab Build Template 下方所述，该方法依赖于 Docker 和 pack CLI。我们试图在我们的 .gitlab-ci.yml 中模仿它，它看起来像这样：

image: docker:20.10.9

stages:
  - build

before_script:
  - |
    echo "install pack CLI (see https://buildpacks.io/docs/tools/pack/)"
    apk add --no-cache curl
    (curl -sSL "https://github.com/buildpacks/pack/releases/download/v0.21.1/pack-v0.21.1-linux.tgz" | tar -C /usr/local/bin/ --no-same-owner -xzv pack)

build-image:
  stage: build
  script:
    - pack --version
    - >
      pack build $REGISTRY_GROUP_PROJECT/$CI_PROJECT_NAME:latest
      --builder paketobuildpacks/builder:base
      --path . 

但如前所述，我们的设置不支持 docker，最终我们的日志中出现以下错误：

...
$ echo "install pack CLI (see https://buildpacks.io/docs/tools/pack/)" # collapsed multi-line command
install pack CLI (see https://buildpacks.io/docs/tools/pack/)
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
(1/4) Installing brotli-libs (1.0.9-r5)
(2/4) Installing nghttp2-libs (1.43.0-r0)
(3/4) Installing libcurl (7.79.1-r0)
(4/4) Installing curl (7.79.1-r0)
Executing busybox-1.33.1-r3.trigger
OK: 12 MiB in 26 packages
pack
$ pack --version
0.21.1+git-e09e397.build-2823
$ pack build $REGISTRY_GROUP_PROJECT/$CI_PROJECT_NAME:latest --builder paketobuildpacks/builder:base --path .
ERROR: failed to build: failed to fetch builder image 'index.docker.io/paketobuildpacks/builder:base': Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: command terminated with exit code 1

关于如何在 GitLab CI 中使用 Paketo Buildpacks 而没有 Docker 出现在我们的 GitLab Kubernetes 运行器中的任何想法（这似乎是一种最佳实践）？我们也不希望我们的设置变得过于复杂——例如通过添加 kpack.

Answer 1

TLDR;

直接在 .gitlab-ci.yml here's a fully working example):

中使用 Buildpack 的生命周期

image: paketobuildpacks/builder

stages:
  - build

# We somehow need to access GitLab Container Registry with the Paketo lifecycle
# So we simply create ~/.docker/config.json as stated in 
before_script:
  - mkdir ~/.docker
  - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_JOB_TOKEN\"}}}" >> ~/.docker/config.json

build-image:
  stage: build
  script:
    - /cnb/lifecycle/creator -app=. $CI_REGISTRY_IMAGE:latest

---

详情：“直接使用生命周期”

关于这个话题的讨论正在进行中。尤其要看看 https://github.com/buildpacks/pack/issues/564 and https://github.com/buildpacks/pack/issues/413#issuecomment-565165832。如那里所述：

If you're looking to build images in CI (not locally), I'd encourage you to use the lifecycle directly for that, so that you don't need Docker. Here's an example:

示例中的 link 已损坏，但它引用了 Tekton implementation on how to use buildpacks in a Kubernetes environment. Here we can get a first glue about what Stephen Levine referred to as "to use the lifecycle directly". Inside it the crucial point is the usage of command: ["/cnb/lifecycle/creator"]. So this is the lifecycle everyone is talking about! And there's good documentaion about this command that could be found in this CNB RFC。

选择好图片：paketobuildpacks/builder:base

那么如何开发一个工作.gitlab-ci.yml？让我们从简单的开始。挖掘 into the Tekton implementation you'll see that the lifecycle command is executed inside an environment defined in BUILDER_IMAGE, which itself is documented as The image on which builds will run (must include lifecycle and compatible buildpacks). That sound's familiar! Can't we simply pick the builder image paketobuildpacks/builder:base from our pack CLI command? Let's try this locally on our workstation before commiting to much noise into our GitLab. Choose a project you want to build (I created a example Spring Boot app if you'd like at gitlab.com/jonashackt/microservice-api-spring-boot 你可以克隆）和 运行:

docker run --rm -it -v "$PWD":/usr/src/app -w /usr/src/app paketobuildpacks/builder bash

现在在 paketobuildpacks/builder 图像驱动的容器内尝试直接 运行 Paketo 生命周期：

/cnb/lifecycle/creator -app=. microservice-api-spring-boot:latest

我只使用了the many possible parameters for the creator command的-app参数，因为它们中的大多数都有很好的默认值。但是由于默认的应用程序目录路径不是默认的 /workspace - 而是当前目录，所以我配置了它。我们还需要在末尾定义一个 <image-name>，它将简单地用作生成的容器镜像名称。

第一个.gitlab-ci.yml

这两个命令都在我的本地工作站上工作，所以让我们最终使用这种方法创建一个 .gitlab-ci.yml (here's a fully working example .gitlab-ci.yml):

image: paketobuildpacks/builder

stages:
  - build

build-image:
  stage: build
  script:
    - /cnb/lifecycle/creator -app=. $CI_REGISTRY_IMAGE:latest

docker 不登录 docker

由于我们的 Kubernetes Runner 中没有可用的 docker，因此我们不能 login into GitLab Container Registry as described in the docs。所以我使用第一种方法时出现了以下错误：

===> ANALYZING
ERROR: failed to get previous image: connect to repo store "gitlab.yourcompanyhere.cloud:4567/yourgroup/microservice-api-spring-boot:latest": GET https://gitlab.yourcompanyhere.cloud/jwt/auth?scope=repository%3Ayourgroup%2Fmicroservice-api-spring-boot%3Apull&service=container_registry: DENIED: access forbidden
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: command terminated with exit code 1

使用方法 fixed the problem. We need to create a ~/.docker/config.json containing the GitLab Container Registry login information - and then the Paketo build will pick them up, as stated in the docs:

If CNB_REGISTRY_AUTH is unset and a docker config.json file is present, the lifecycle SHOULD use the contents of this file to authenticate with any matching registry.

在我们的 .gitlab-ci.yml 中，它看起来像：

# We somehow need to access GitLab Container Registry with the Paketo lifecycle
# So we simply create ~/.docker/config.json as stated in 
before_script:
  - mkdir ~/.docker
  - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_JOB_TOKEN\"}}}" >> ~/.docker/config.json

我们最终的.gitlab-ci.yml

由于我们在 .gitlab-ci.yml 的顶部使用 image: paketobuildpacks/builder，我们现在可以直接利用生命周期。这正是我们最初想要做的。只记得使用正确的 GitLab CI 变量来描述你的 <image-name> 像这样：

/cnb/lifecycle/creator -app=. $CI_REGISTRY_IMAGE:latest

否则 Buildpack 流程分析器步骤将中断，并且它最终不会被推送到 GitLab 容器注册表。所以最后我们的 .gitlab-ci.yml 看起来像这样 (here's the fully working example):

image: paketobuildpacks/builder

stages:
  - build

# We somehow need to access GitLab Container Registry with the Paketo lifecycle
# So we simply create ~/.docker/config.json as stated in 
before_script:
  - mkdir ~/.docker
  - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_JOB_TOKEN\"}}}" >> ~/.docker/config.json

build-image:
  stage: build
  script:
    - /cnb/lifecycle/creator -app=. $CI_REGISTRY_IMAGE:latest

我们的构建现在应该 运行 成功使用 Paketo/Buildpacks 没有 pack CLI 和 Docker:

参见 the full log of the example project here。