如何安全地 git clone/pip 将私有存储库安装到我的 docker 映像中？

Question

我有一个私人存储库，其中包含我要 pip 安装的软件包。我花了很多时间阅读有关安全执行此操作的不同方法的各种论坛和文章。对于如何最好地做到这一点（如果可能的话）似乎没有达成共识。我显然不想在我的 docker 文件中公开任何 ssh keys/secrets -- 我想小心地通过 docker 历史使它们可用。

Answer 1

如“Securely build small python docker image from private git repos”中所述，您需要使用 Docker 18.09+

--ssh
You can use the --ssh flag to forward your existing SSH agent key to the builder. Instead of transferring the key data, docker will just notify the builder that such capability is available.
Now when the builder needs access to a remote server through SSH, it will dial back to the client and ask it to sign the specific request needed for this connection.
The key itself never leaves the client, and as soon as the command that requested the access has completed there is no information on the builder side to reestablish that remote connection later.

Secrets:
Provides a mount option during the build at /var/run/secrets available only for the command that used it and is not included in the created layer.

即：

docker build --ssh github_ssh_key=/path/to/.ssh/git_ssh_id_rsa .

only the agent connection is shared with that command, and not the actual content of the private key.
no other commands/steps in the Dockerfile will have access to it.

Docker文件，在多阶段的第一步中，会给出一个键名github_ssh_key，这样我们就可以在调用docker build:

时使用它

RUN --mount=type=ssh,id=github_ssh_key pip wheel \
    --no-cache \
    --requirement requirements.txt \
--wheel-dir=/app/wheels

OP Jesus Garcia 确实报告（在评论中）使它起作用：

I had to use 2 separate RUN commands.

I'm not sure if it's a limitation of this new feature, or the way I was trying to string together multiple commands in my RUN but I kept getting a publickey permission denied error when I added it as other commands && /bin/sh -c "mount=type=ssh,id=github_ssh_key pip install private-repo" vs RUN --mount=type=ssh,id=github_ssh_key pip install private-repo && more commands ...

Answer 2

这对我适用于 docker 19.03.5（但应该适用于 18.09+）：

假设您的私人仓库在 Github、github.com/user/repo.git 上，并且您想 pip install 它作为 docker 构建的一部分。存储库的 ssh 密钥位于主机上 ~/.ssh/my_key.

在 Dockerfile 中：

RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh pip install git+ssh://git@github.com/user/repo.git

然后，在主机上：

export DOCKER_BUILDKIT=1
eval `ssh-agent`
ssh-add ~/.ssh/my_key
docker build --ssh default=$SSH_AUTH_SOCK .

Answer 3

尽管使用现代 (19.03.8) Docker，但如果没有我在 Alexandra Ulsh's blog post 中找到的额外行，上述两种解决方案都不适合我（即使在前面加上 DOCKER_BUILDKIT=1 之后），我认为这足够重要，值得一个额外的答案，而不是被埋在评论中。

Though the --secret flag appears in a production version of Docker, there are a few hints that support is still somewhat experimental.

For example, the first line of a Dockerfile using build secrets must be # syntax = docker/dockerfile:1.0-experimental. Without this line you’ll get the error failed to create LLB definition: Dockerfile parse error line 6: Unknown flag: mount. This line enables the Docker CLI to use the “experimental Dockerfile frontend” for Moby BuildKit.

TL;DR 在 Docker 文件的顶部添加注释 # syntax = docker/dockerfile:1.0-experimental，然后跟随 .

Answer 4

我使用的是 Docker 版本 19.03.13（内部版本 4484c46d9d），使用 and 答案中描述的 --ssh 功能对我不起作用。我仍然不断收到以下错误：

ERROR: Command errored out with exit status 128: git clone -q 'ssh://git@hostname:port/group/repository.git' /tmp/pip-install-8eeeaipo/baseapi Check the logs for full command output.

请注意，我不是 pip install-ing 来自 Github 存储库，而是来自某些 really private/internal 存储库。 Docker docs on --ssh 中的示例还显示了“github.com”回购 URL。我不确定是否是这种情况（不应该是），但我已经追踪到 pip/git 无法识别 URL 的主机名，而如果使用“github.com”它像宣传的那样工作。

我通过传入 SSH 密钥 (~/.ssh/id_rsa) 和 ~/.ssh/known_hosts 文件，仅使用普通的 --secret 函数（仍然是 BuildKit 的一部分）使其工作作为秘密。

Docker文件

# syntax=docker/dockerfile:1.0-experimental

FROM python:3.8-alpine3.11

RUN apk add openssh-client git
RUN mkdir -p -m 0600 /root/.ssh
RUN --mount=type=secret,id=known_hosts,dst=/root/.ssh/known_hosts \
    --mount=type=secret,id=ssh_key,dst=/root/.ssh/id_rsa \
    ssh-keyscan -H -t rsa hostname.of.private.repo \
    && pip install --no-cache-dir git+ssh://git@hostname:port/group/repository.git@12345678#egg=baseapi

生成命令

DOCKER_BUILDKIT=1 docker build \
    --secret id=known_hosts,src=~/.ssh/known_hosts \
    --secret id=ssh_key,src=~/.ssh/id_rsa \
    -t name:tag \
    -f Dockerfile \
    .

我检查了构建映像上的 ~/.ssh 文件夹，它似乎仍然正确，它不会保留 SSH 密钥。挂载的文件仍然存在，但它们是空的：

/app # ls -l ~/.ssh/
total 0
-rwxr-xr-x    1 root     root             0 Nov 19 11:29 id_rsa
-rwxr-xr-x    1 root     root             0 Nov 19 11:29 known_hosts
/app # cat ~/.ssh/id_rsa 
/app # cat ~/.ssh/known_hosts

并且尝试 pip install 同一个私有包应该会失败。

供参考：

* Docker: 19.03.13 (build 4484c46d9d)
* Image
    - pip 20.2.4 from /usr/local/lib/python3.8/site-packages/pip (python 3.8)
    - git 2.22.4
    - OpenSSH_8.1p1, OpenSSL 1.1.1g  21 Apr 2020

Answer 5

使用RUN --mount=type=ssh

ci.yml

build_docker:
  stage: build
  image: docker:latest
  variables:
    DOCKER_HOST: tcp://docker:2375
    # This instructs Docker not to start over TLS.
    DOCKER_TLS_CERTDIR: ""

  before_script:
    - apk add --no-cache curl jq python3 py3-pip git bind-tools openssh-client
    - echo $DOCKERPASS | docker login --username $DOCKERUSER --password-stdin
    - mkdir -p -m 0700 ~/.ssh
    - ssh-keyscan private-url >> ~/.ssh/known_hosts 
    - chmod 600 $SSH_KEY

  services:
    - docker:dind

  script:
    - export DOCKER_BUILDKIT=1
    - eval $(ssh-agent)
    - ssh-add $SSH_KEY
    - docker build -t tag_name  --ssh default=$SSH_AUTH_SOCK .

dockerfile

FROM puckel/docker-airflow:1.10.1

ARG SSH_PRIVATE_KEY

USER root
RUN apt-get update && apt-get install -y git openssh-client


RUN ssh-keyscan private-url >> ~/.ssh/known_hosts

COPY requirements.txt .

RUN --mount=type=ssh pip install git+ssh://private-url/project/repo.git
RUN pip install -r requirements.txt

看图https://feiyang233.club/img/vm/23.png

如何安全地 git clone/pip 将私有存储库安装到我的 docker 映像中？

How to securely git clone/pip install a private repository into my docker image?

git

ssh-keys

docker-image