gitlab-runner 在创建挂载源路径 '/usr/share/ca-certificates/mozilla' 时出现 kubernetes 集群错误

Question

我正在尝试在 kubernetes 集群上获取 gitlab-runner "run"，遵循官方文档 -> https://docs.gitlab.com/runner/install/kubernetes.html（使用 kubernetes 执行器）我在部署后遇到错误:

Error: failed to start container "gitlab-runner": Error response from daemon: error while creating mount source path '/usr/share/ca-certificates/mozilla': mkdir /usr/share/ca-certificates/mozilla: read-only file system

我正在使用该网站中的示例，但无法弄清楚为什么不允许创建该目录（据我所知，默认用户是 root）

这是我的配置-map.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: gitlab-runner
  namespace: gitlab
data:
  config.toml: |
    concurrent = 1
    [[runners]]
      name = "Kubernetes Runner"
      url = "URL"
      token = "TOKEN"
      executor = "kubernetes"
      [runners.kubernetes]
        namespace = "gitlab"

这是 deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gitlab-runner
  namespace: gitlab
spec:
  replicas: 1
  selector:
    matchLabels:
      name: gitlab-runner
  template:
    metadata:
      labels:
        name: gitlab-runner
    spec:
      containers:
        - args:
            - run
          image: gitlab/gitlab-runner:alpine-v11.5.0
          imagePullPolicy: Always
          name: gitlab-runner
          volumeMounts:
            - mountPath: /etc/gitlab-runner
              name: config
            - mountPath: /etc/ssl/certs
              name: cacerts
              readOnly: true
      restartPolicy: Always
      volumes:
        - configMap:
            name: gitlab-runner
          name: config
        - hostPath:
            path: /usr/share/ca-certificates/mozilla
          name: cacerts

这是初始化 pod 的完整事件列表：

Events:
  Type     Reason                 Age                From                                                          Message
  ----     ------                 ----               ----                                                          -------
  Normal   Scheduled              29s                default-scheduler                                             Successfully assigned gitlab-runner-5b689c7cbc-hw6r5 to gke-my-project-dev-default-pool-0d32b263-6skk
  Normal   SuccessfulMountVolume  29s                kubelet, gke-my-project-dev-default-pool-0d32b263-6skk  MountVolume.SetUp succeeded for volume "cacerts"
  Normal   SuccessfulMountVolume  29s                kubelet, gke-my-project-dev-default-pool-0d32b263-6skk  MountVolume.SetUp succeeded for volume "config"
  Normal   SuccessfulMountVolume  29s                kubelet, gke-my-project-dev-default-pool-0d32b263-6skk  MountVolume.SetUp succeeded for volume "default-token-6hr2h"
  Normal   Pulling                23s (x2 over 28s)  kubelet, gke-my-project-dev-default-pool-0d32b263-6skk  pulling image "gitlab/gitlab-runner:alpine-v11.5.0"
  Normal   Pulled                 19s (x2 over 24s)  kubelet, gke-my-project-dev-default-pool-0d32b263-6skk  Successfully pulled image "gitlab/gitlab-runner:alpine-v11.5.0"
  Normal   Created                19s (x2 over 24s)  kubelet, gke-my-project-dev-default-pool-0d32b263-6skk  Created container
  Warning  Failed                 19s (x2 over 24s)  kubelet, gke-my-project-dev-default-pool-0d32b263-6skk  Error: failed to start container "gitlab-runner": Error response from daemon: error while creating mount source path '/usr/share/ca-certificates/mozilla': mkdir /usr/share/ca-certificates/mozilla: read-only file system
  Warning  BackOff                14s                kubelet, gke-my-project-dev-default-pool-0d32b263-6skk  Back-off restarting failed container

任何线索将不胜感激

谢谢

Answer 1

根据日志，我猜您正在使用 GKE。 Google 安全安装您的 / 文件系统（参见 here）。这就是你出错的原因。

通过启用容器的 privileged 模式来尝试：

containers:
   securityContext:
     privileged: true

如果这不起作用，则将 /usr/share/ca-certificates/mozilla 更改为 /var/SOMETHING（不确定，这是一个很好的解决方案）。如果 /usr/share/ca-certificates/mozilla 中有文件，则 move/copy 到 /var/SOMETHING

Answer 2

最后，我在这里使用 register 和运行 gitlab-[=28= GKE 上的 ]ner

配置图：

apiVersion: v1
kind: ConfigMap
metadata:
  name: gitlab-runner-cm
  namespace: gitlab
data:
  config.toml: |
    concurrent = 4
    check_interval = 30
  entrypoint: |
    #!/bin/bash

    set -xe
    cp /scripts/config.toml /etc/gitlab-runner/

    # Register the runner
    /entrypoint register --non-interactive \
      --url $GITLAB_URL \
      --tag-list "kubernetes, my_project" \
      --kubernetes-image "alpine:latest" \
      --kubernetes-namespace "gitlab" \
      --executor kubernetes \
      --config "/etc/gitlab-runner/config.toml" \
      --locked=false \
      --run-untagged=true \
      --description "My Project - Kubernetes Runner" \
      --kubernetes-privileged

    # Start the runner
    /entrypoint run --user=gitlab-runner \
      --working-directory=/home/gitlab-runner \
      --config "/etc/gitlab-runner/config.toml"

部署：

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gitlab-runner
  namespace: gitlab
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gitlab-runner
  template:
    metadata:
      labels:
        app: gitlab-runner
    spec:
      containers:
        - name: gitlab-runner
          image: gitlab/gitlab-runner:latest
          command: ["/bin/bash", "/scripts/entrypoint"]
          env:
            - name: GITLAB_URL
              value: "URL"
            - name: REGISTRATION_TOKEN
              value: "TOKEN"
            - name: KUBERNETES_NAMESPACE
              value: gitlab
            - name: GOOGLE_APPLICATION_CREDENTIALS
              value: /var/secrets/google/key.json
          imagePullPolicy: Always
          volumeMounts:
            - name: config
              mountPath: /scripts
            - name: google-cloud-key
              mountPath: /var/secrets/google
      restartPolicy: Always
      volumes:
        - name: config
          configMap:
            name: gitlab-runner-cm
        - name: google-cloud-key
          secret:
            secretName: gitlab-runner-sa

和自动缩放：

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: gitlab-runner-hpa
  namespace: gitlab
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: gitlab-runner
  minReplicas: 1
  maxReplicas: 3
  metrics:
    - type: Resource
      resource:
        name: cpu
        targetAverageUtilization: 50

我希望这有助于尝试在 Google Kubernetes Engine

上的 Kubernetes 集群中运行 Gitlab Runner

gitlab-runner 在创建挂载源路径 '/usr/share/ca-certificates/mozilla' 时出现 kubernetes 集群错误

gitlab-runner on a kubernetes cluster error while creating mount source path '/usr/share/ca-certificates/mozilla'

kubernetes

gitlab-ci-runner

google-kubernetes-engine