GitLab CI runner 无法连接到 unix:///var/run/docker.sock in kubernetes

Question

GitLab 的运行在 kubernetes 集群中。 Runner 无法使用构建工件构建 docker 图像。我已经尝试了几种方法来解决这个问题，但没有成功。以下是一些配置片段：

.gitlab-ci.yml

image: docker:latest
services:
  - docker:dind

variables:
  DOCKER_DRIVER: overlay

stages:
  - build
  - package
  - deploy

maven-build:
  image: maven:3-jdk-8
  stage: build
  script: "mvn package -B --settings settings.xml"
  artifacts:
    paths:
      - target/*.jar

docker-build:
  stage: package
  script:
  - docker build -t gitlab.my.com/group/app .
  - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN gitlab.my.com/group/app
  - docker push gitlab.my.com/group/app

config.toml

concurrent = 1
check_interval = 0

[[runners]]
  name = "app"
  url = "https://gitlab.my.com/ci"
  token = "xxxxxxxx"
  executor = "kubernetes"
  [runners.kubernetes]
    privileged = true
    disable_cache = true

包阶段日志：

running with gitlab-ci-multi-runner 1.11.1 (a67a225)
  on app runner (6265c5)
Using Kubernetes namespace: default
Using Kubernetes executor with image docker:latest ...
Waiting for pod default/runner-6265c5-project-4-concurrent-0h9lg9 to be running, status is Pending
Waiting for pod default/runner-6265c5-project-4-concurrent-0h9lg9 to be running, status is Pending
Running on runner-6265c5-project-4-concurrent-0h9lg9 via gitlab-runner-3748496643-k31tf...
Cloning repository...
Cloning into '/group/app'...
Checking out 10d5a680 as master...
Skipping Git submodules setup
Downloading artifacts for maven-build (61)...
Downloading artifacts from coordinator... ok        id=61 responseStatus=200 OK token=ciihgfd3W
$ docker build -t gitlab.my.com/group/app .
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
ERROR: Job failed: error executing remote command: command terminated with non-zero exit code: Error executing in Docker Container: 1

我做错了什么？

Answer 1

使用 Kubernetes 时，您必须调整构建映像以连接 Docker 引擎。

添加到您的构建映像：

DOCKER_HOST=tcp://localhost:2375

引自文档：

Running the docker:dind also known as the docker-in-docker image is also possible but sadly needs the containers to be run in privileged mode. If you're willing to take that risk other problems will arise that might not seem as straight forward at first glance. Because the docker daemon is started as a service usually in your .gitlab-ci.yaml it will be run as a separate container in your pod. Basically containers in pods only share volumes assigned to them and an IP address by wich they can reach each other using localhost. /var/run/docker.sock is not shared by the docker:dind container and the docker binary tries to use it by default. To overwrite this and make the client use tcp to contact the docker daemon in the other container be sure to include DOCKER_HOST=tcp://localhost:2375 in your environment variables of the build container.

Gitlab-CI on Kubernetes

Answer 2

不需要使用这个：

DOCKER_DRIVER: overlay

因为似乎不支持 OVERLAY，所以 svc-0 容器无法用它启动：

$ kubectl logs -f `kubectl get pod |awk '/^runner/{print }'` -c svc-0
time="2017-03-20T11:19:01.954769661Z" level=warning msg="[!] DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING [!]"
time="2017-03-20T11:19:01.955720778Z" level=info msg="libcontainerd: new containerd process, pid: 20"
time="2017-03-20T11:19:02.958659668Z" level=error msg="'overlay' not found as a supported filesystem on this host. Please ensure kernel is new enough and has overlay support loaded."

此外，将 export DOCKER_HOST="tcp://localhost:2375" 添加到 docker-build:

 docker-build:
  stage: package
  script:
  - export DOCKER_HOST="tcp://localhost:2375"
  - docker build -t gitlab.my.com/group/app .
  - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN gitlab.my.com/group/app
  - docker push gitlab.my.com/group/app

Answer 3

根据@Yarik 的评论，对我有用的是

- export DOCKER_HOST=$DOCKER_PORT

没有其他答案有效。

Answer 4

我有同样的问题，我无法让上述解决方法为我工作（我没有尝试@fkpwolf 提到的卷技巧）。

现在 GitLab 有一个使用 Kaniko 的替代解决方案，它对我有用：

然后 .gitlab-ci.yaml 可能是这样的，在那种情况下：

stages:
  - build
  - package
  - deploy

maven-build:
  image: maven:3-jdk-8
  stage: build
  script: "mvn package -B --settings settings.xml"
  artifacts:
    paths:
      - target/*.jar

docker-kaniko-build:
  stage: package
  image:
    name: gcr.io/kaniko-project/executor:debug
    entrypoint: [""]
  script:
    - echo "{\"auths\":{\"gitlab.my.com\":{\"username\":\"gitlab-ci-token\",\"password\":\"$CI_BUILD_TOKEN\"}}}" > /kaniko/.docker/config.json
    - /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination gitlab.my.com/group/app

从 GitLab 文档中提到：

kaniko solves two problems with using the docker-in-docker build method:

Docker-in-docker requires privileged mode in order to function, which is a significant security concern.

Docker-in-docker generally incurs a performance penalty and can be quite slow.

参见：https://docs.gitlab.com/ee/ci/docker/using_kaniko.html

GitLab CI runner 无法连接到 unix:///var/run/docker.sock in kubernetes

GitLab CI runner can't connect to unix:///var/run/docker.sock in kubernetes

gitlab

docker

kubernetes

gitlab-ci-runner