Kaniko：如何使用 Tekton 在 Kubernetes 中缓存来自 Gatsby 的文件夹？

Question

我正在裸机 Kubernetes 集群上使用 Tekton 构建 CI/CD 管道。我已经设法缓存了必要的图像（Node 和 Nginx）和图层，但是我如何缓存由 Gatsby build 创建的 .cache / public 文件夹？这些文件夹不存在于 repo 中。如果构建步骤没有找到这些文件夹，则需要更长的时间，因为它需要使用 Sharp 创建所有图像。

管道连接了 PVC。在任务中它被称为 source （工作区）。更清楚地说，如何在构建完成后将 Gatsby 文件夹复制到此 PVC，并在下一次构建之前复制到 Kaniko 容器？

Tekton任务有以下步骤：

使用 Kaniko warmer 缓存 Docker Docker build
创建一个时间戳，以便每次执行“运行构建”，即使文件没有更改，因为它运行 GraphQL 查询
使用 Kaniko 构建和推送镜像
& 5.导出管道中下一步使用的图像摘要

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: build-docker-image
spec:
  params:
    - name: pathToDockerFile
      type: string
      description: The path to the dockerfile to build
      default: $(resources.inputs.source-repo.path)/Dockerfile
    - name: pathToContext
      type: string
      description: |
        The build context used by Kaniko
        (https://github.com/GoogleContainerTools/kaniko#kaniko-build-contexts)
      default: $(resources.inputs.source-repo.path)
  resources:
    inputs:
      - name: source-repo
        type: git
    outputs:
      - name: builtImage
        type: image
      - name: event-to-sink
        type: cloudEvent
  workspaces:
    # PVC
    - name: source
      description: |
        Folder to write docker image digest
  results:
    - name: IMAGE-DIGEST
      description: Digest of the image just built.
  steps:
    - name: kaniko-warmer
      image: gcr.io/kaniko-project/warmer
      workingDir: $(workspaces.source.path)
      args:
        - --cache-dir=$(workspaces.source.path)/cache
        - --image=node:14-alpine
        - --image=nginx:1.19.5
    - name: print-date-unix-timestamp
      image: bash:latest
      script: |
        #!/usr/bin/env bash
        date | tee $(params.pathToContext)/date
    - name: build-and-push
      workingDir: $(workspaces.source.path)
      image: gcr.io/kaniko-project/executor:v1.3.0
      env:
        - name: 'DOCKER_CONFIG'
          value: '/tekton/home/.docker/'
      command:
        - /kaniko/executor
      args:
        - --build-arg=CACHEBUST=$(params.pathToContext)/date
        - --dockerfile=$(params.pathToDockerFile)
        - --destination=$(resources.outputs.builtImage.url)
        - --context=$(params.pathToContext)
        - --cache=true
        - --cache-ttl=144h
        - --cache-dir=$(workspaces.source.path)/cache
        - --use-new-run
        - --snapshotMode=redo
        - --cache-repo=<repo>/kaniko-cache
        - --log-timestamp
      securityContext:
        runAsUser: 0
    - name: write-digest
      workingDir: $(workspaces.source.path)
      image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/imagedigestexporter:v0.16.2
      command: ['/ko-app/imagedigestexporter']
      args:
        - -images=[{"name":"$(resources.outputs.builtImage.url)","type":"image","url":"$(resources.outputs.builtImage.url)","digest":"","OutputImageDir":"$(workspaces.source.path)/$(params.pathToContext)/image-digest"}]
        - -terminationMessagePath=$(params.pathToContext)/image-digested
      securityContext:
        runAsUser: 0
    - name: digest-to-result
      workingDir: $(workspaces.source.path)
      image: docker.io/stedolan/jq@sha256:a61ed0bca213081b64be94c5e1b402ea58bc549f457c2682a86704dd55231e09
      script: |
        cat $(params.pathToContext)/image-digested | jq '.[0].value' -rj | tee /$(results.IMAGE-DIGEST.path)

Docker文件

FROM node:14-alpine as build
ARG CACHEBUST=1

RUN apk update \
  && apk add \
  build-base \
  libtool \
  autoconf \
  automake \
  pkgconfig \
  nasm \
  yarn \
  libpng-dev libjpeg-turbo-dev giflib-dev tiff-dev \
  zlib-dev \
  python \
  && rm -rf /var/cache/apk/*

EXPOSE 8000 9000

RUN yarn global add gatsby-cli

WORKDIR /usr/src/app
COPY ./package.json .
RUN yarn install
COPY . .
RUN yarn build && echo $CACHEBUST

CMD ["yarn", "serve"]

FROM nginx:1.19.5 as serve
EXPOSE 80
COPY --from=build /usr/src/app/public /usr/share/nginx/html

Answer 1

how can I cache the .cache / public folders created by Gatsby build? These folders are not present in the repo.

如果持久卷在您的集群上可用，并且这些卷在所有节点上都可用，您可以使用 PVC 支持的工作区进行缓存。

也适用于区域集群（例如云）的更通用的解决方案是将缓存的文件夹上传到某个地方，例如一个桶（Minio?) or potentially Redis? Then also need a Task that download this folder - potentially in parallel with git clone when starting a new PipelineRun. GitHub Actions has a similar solution with the cache action.

具有两个工作区的任务示例，将文件从一个工作区复制到另一个工作区：

apiVersion: tekton.dev/v1beta1  
kind: Task  
metadata:   
  name: copy-between-workspaces 
spec:   
  workspaces:   
    - name: ws-a    
    - name: ws-b    
  steps:    
    - name: copy
      image: ubuntu 
      script: cp $(workspaces.ws-a.path)/myfile $(workspaces.ws-b.path)/myfile

Kaniko：如何使用 Tekton 在 Kubernetes 中缓存来自 Gatsby 的文件夹？

Kaniko: How to cache folders from Gatsby build in Kubernetes using Tekton?

docker

kubernetes

kaniko

tekton

tekton-pipelines