Gitlab CI 服务和 docker 集线器身份验证
Gitlab CI service and docker hub auth
由于 new restrictions 在 docker 集线器上未经身份验证的拉取，您如何为 gitlab-ci 服务验证您的 docker 集线器帐户？
这是来自 the gitlab documentation 的示例 CI 配置：
# from official documentation
services:
- postgres:12.2 # <---- this will fail at some point because it's a non-authenticated pull
variables:
POSTGRES_DB: nice_marmot
POSTGRES_USER: runner
POSTGRES_PASSWORD: ""
POSTGRES_HOST_AUTH_METHOD: trust
这会在一段时间后导致以下错误：
ERROR: Preparation failed: Error response from daemon:
toomanyrequests: You have reached your pull rate limit.
You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit (executor_docker.go:188:1s)
由于服务是在脚本 运行 之前拉取的，我们不能在脚本部分 docker login。我无法从 gitlab 中找到任何关于 url auth 或环境变量 auth.
的文档
一个理想的解决方案不需要拥有对 gitlab-ci 服务器或 gitlab-ci 运行 用户的管理员访问权限，也不需要设置自定义 运行ner 与 pull_policy = never（我们最终这样做了，但它大大减慢了我们的 CI 速度，因为 e2e 测试有一个 运行ner 瓶颈）
我遇到了同样的错误，并查看了 Docker documentation on toomanyrequests, that provides a way to retrieve a token, that can then be used in Gitlab-CI。
但是，我还不清楚：我只是尝试了第一步，并在尝试获取剩余配额时得到 401 unauthorized :-/
另一条有趣的路径：我们可能必须define our credentials for Docker hub as for a private repo。但是，设置具有多行内容的变量和 " 等字符使其不可屏蔽，考虑到我们仅提供 username:password base-64 编码而不是加密，这似乎是一个问题...... :-/
另请检查 GitLab 13.7（2020 年 12 月）改进的依赖代理是否可以提供帮助：
Avoid Docker rate limits and speed up your pipelines
For faster and more reliable builds, you can use the Dependency Proxy to cache the container images hosted on Docker Hub.
But, when Docker started to enforce rate limits on pull requests from Docker Hub, you noticed that even when your image was pulled from the cache, Docker counted it against your limit.
That’s because the Dependency Proxy was only caching the image’s layers (or blobs) and not the manifest, which contains information about how to build a given image.
Since the manifest is required, a pull request was still required. This also means that if Docker Hub was unavailable, you couldn’t pull your image.
Moving forward, the Dependency Proxy will cache both the image’s layers and manifest.
So, the first time you pull alpine:latest, the image will be added to the Dependency Proxy cache and count as one pull against your rate limit.
The next time you pull alpine:latest, it will be pulled from the cache, even if Docker Hub is unavailable and will not count against your rate limit.
Don’t forget, as of milestone 13.6, the Dependency Proxy is available in Core. So, give it a try and let us know what you think. Or better yet, consider contributing to one of the open issues.
See Documentation and Issue.
并且：
仍然 GitLab 13.7（2020 年 12 月）
Use pre-defined variables with the Dependency Proxy
By proxying and caching container images from Docker Hub, the Dependency Proxy helps you to improve the performance of your pipelines.
Even though the proxy is intended to be heavily used with CI/CD, to use the feature, you had to define your own variables or hard-code values in your gitlab.ci-yml file.
This made it difficult to get started for individuals, and prevented it from being a scalable solution, especially for organizations with many different groups and projects.
Moving forward, you can use pre-defined environment variables as an intuitive way to use the Dependency Proxy. The following variables are supported:
CI_DEPENDENCY_PROXY_USER: a CI user for logging in to the Dependency Proxy.CI_DEPENDENCY_PROXY_PASSWORD: a CI password for logging in to the Dependency Proxy.CI_DEPENDENCY_PROXY_SERVER: the server for logging in to the Dependency Proxy.CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX: the image prefix for pulling images through the Dependency Proxy.
Give it a try and let us know what you think!
See Documentation and Issue.
这甚至适用于 private projects（2020 年 12 月）
Use the Dependency Proxy with private projects
You can use the GitLab Dependency Proxy to proxy and cache container images from Docker Hub. Until recently the feature was only available for public groups, preventing many of you from being able to use it.
You can now use the Dependency Proxy with private projects.
You can reduce your reliance on Docker Hub by caching your container images for future use.
Because the Dependency Proxy is storing Docker images in a space associated with your group, you must authenticate with your GitLab username and password, or with your personal access token with the scope set to at least read_registry.
See Documentation and Issue.
GitLab 13.9（2021 年 2 月）：
Automatically authenticate when using the Dependency Proxy
By proxying and caching container images from Docker Hub, the Dependency Proxy helps you to improve the performance of your pipelines.
Even though the proxy is intended to be heavily used with CI/CD, to use the feature, you had to add your credentials to the DOCKER_AUTH_CONFIG CI/CD variable or manually run docker login in your pipeline. These solutions worked fine, but when you consider how many .gitlab-ci.yml files that you need to update, it would be better if the GitLab Runner could automatically authenticate for you.
Since the Runner is already able to automatically authenticate with the integrated GitLab Container Registry, we were able to leverage that functionality to help you automatically authenticate with the Dependency Proxy.
Now it’s easier to use the Dependency Proxy to proxy and cache your container images from Docker Hub and start having faster, more reliable builds.
See Documentation and Issue.
参见 GitLab 13.10（2021 年 3 月）
Use the Dependency Proxy with 'containerd' and Docker 20+
The GitLab Dependency Proxy is a local proxy you can use for your frequently-accessed upstream images from Docker Hub. In the case of CI/CD, the Dependency Proxy receives a request and returns the upstream image from a registry, acting as a pull-through cache. This helps to reduce your CI minutes and increase reliability.
However, you haven’t been able to pull images by digest, which as an immutable identifier ensures you are using the exact version of a specific image and tag. Since both containerd and Docker 20+ depend on pull-by-digest, this meant that many of you were blocked from using the Dependency Proxy.
We are happy to say that you can now pull your container images from Docker Hub by digest. You can use the Dependency Proxy by adding the URL to your .gitlab-ci.yml file, manually pulling the image from the command line, or using a Dockerfile. Check out the documentation and start saving time on your builds.
See Documentation and Issue.
由于 new restrictions 在 docker 集线器上未经身份验证的拉取，您如何为 gitlab-ci 服务验证您的 docker 集线器帐户？
这是来自 the gitlab documentation 的示例 CI 配置：
# from official documentation
services:
- postgres:12.2 # <---- this will fail at some point because it's a non-authenticated pull
variables:
POSTGRES_DB: nice_marmot
POSTGRES_USER: runner
POSTGRES_PASSWORD: ""
POSTGRES_HOST_AUTH_METHOD: trust
这会在一段时间后导致以下错误：
ERROR: Preparation failed: Error response from daemon:
toomanyrequests: You have reached your pull rate limit.
You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit (executor_docker.go:188:1s)
由于服务是在脚本 运行 之前拉取的，我们不能在脚本部分 docker login。我无法从 gitlab 中找到任何关于 url auth 或环境变量 auth.
一个理想的解决方案不需要拥有对 gitlab-ci 服务器或 gitlab-ci 运行 用户的管理员访问权限，也不需要设置自定义 运行ner 与 pull_policy = never（我们最终这样做了，但它大大减慢了我们的 CI 速度，因为 e2e 测试有一个 运行ner 瓶颈）
我遇到了同样的错误，并查看了 Docker documentation on toomanyrequests, that provides a way to retrieve a token, that can then be used in Gitlab-CI。
但是，我还不清楚：我只是尝试了第一步，并在尝试获取剩余配额时得到 401 unauthorized :-/
另一条有趣的路径：我们可能必须define our credentials for Docker hub as for a private repo。但是，设置具有多行内容的变量和 " 等字符使其不可屏蔽，考虑到我们仅提供 username:password base-64 编码而不是加密，这似乎是一个问题...... :-/
另请检查 GitLab 13.7（2020 年 12 月）改进的依赖代理是否可以提供帮助：
Avoid Docker rate limits and speed up your pipelines
For faster and more reliable builds, you can use the Dependency Proxy to cache the container images hosted on Docker Hub.
But, when Docker started to enforce rate limits on pull requests from Docker Hub, you noticed that even when your image was pulled from the cache, Docker counted it against your limit.
That’s because the Dependency Proxy was only caching the image’s layers (or blobs) and not the manifest, which contains information about how to build a given image.
Since the manifest is required, a pull request was still required. This also means that if Docker Hub was unavailable, you couldn’t pull your image.Moving forward, the Dependency Proxy will cache both the image’s layers and manifest.
So, the first time you pull
alpine:latest, the image will be added to the Dependency Proxy cache and count as one pull against your rate limit.
The next time you pullalpine:latest, it will be pulled from the cache, even if Docker Hub is unavailable and will not count against your rate limit.Don’t forget, as of milestone 13.6, the Dependency Proxy is available in Core. So, give it a try and let us know what you think. Or better yet, consider contributing to one of the open issues.
See Documentation and Issue.
并且：
仍然 GitLab 13.7（2020 年 12 月）
Use pre-defined variables with the Dependency Proxy
By proxying and caching container images from Docker Hub, the Dependency Proxy helps you to improve the performance of your pipelines.
Even though the proxy is intended to be heavily used with CI/CD, to use the feature, you had to define your own variables or hard-code values in your
gitlab.ci-ymlfile.
This made it difficult to get started for individuals, and prevented it from being a scalable solution, especially for organizations with many different groups and projects.Moving forward, you can use pre-defined environment variables as an intuitive way to use the Dependency Proxy. The following variables are supported:
CI_DEPENDENCY_PROXY_USER: a CI user for logging in to the Dependency Proxy.CI_DEPENDENCY_PROXY_PASSWORD: a CI password for logging in to the Dependency Proxy.CI_DEPENDENCY_PROXY_SERVER: the server for logging in to the Dependency Proxy.CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX: the image prefix for pulling images through the Dependency Proxy.Give it a try and let us know what you think!
See Documentation and Issue.
这甚至适用于 private projects（2020 年 12 月）
Use the Dependency Proxy with private projects
You can use the GitLab Dependency Proxy to proxy and cache container images from Docker Hub. Until recently the feature was only available for public groups, preventing many of you from being able to use it.
You can now use the Dependency Proxy with private projects.
You can reduce your reliance on Docker Hub by caching your container images for future use.Because the Dependency Proxy is storing Docker images in a space associated with your group, you must authenticate with your GitLab username and password, or with your personal access token with the scope set to at least
read_registry.See Documentation and Issue.
GitLab 13.9（2021 年 2 月）：
Automatically authenticate when using the Dependency Proxy
By proxying and caching container images from Docker Hub, the Dependency Proxy helps you to improve the performance of your pipelines.
Even though the proxy is intended to be heavily used with CI/CD, to use the feature, you had to add your credentials to the
DOCKER_AUTH_CONFIGCI/CD variable or manually rundocker loginin your pipeline. These solutions worked fine, but when you consider how many.gitlab-ci.ymlfiles that you need to update, it would be better if the GitLab Runner could automatically authenticate for you.Since the Runner is already able to automatically authenticate with the integrated GitLab Container Registry, we were able to leverage that functionality to help you automatically authenticate with the Dependency Proxy.
Now it’s easier to use the Dependency Proxy to proxy and cache your container images from Docker Hub and start having faster, more reliable builds.
See Documentation and Issue.
参见 GitLab 13.10（2021 年 3 月）
Use the Dependency Proxy with 'containerd' and Docker 20+
The GitLab Dependency Proxy is a local proxy you can use for your frequently-accessed upstream images from Docker Hub. In the case of CI/CD, the Dependency Proxy receives a request and returns the upstream image from a registry, acting as a pull-through cache. This helps to reduce your CI minutes and increase reliability.
However, you haven’t been able to pull images by digest, which as an immutable identifier ensures you are using the exact version of a specific image and tag. Since both
containerdand Docker 20+ depend on pull-by-digest, this meant that many of you were blocked from using the Dependency Proxy.We are happy to say that you can now pull your container images from Docker Hub by digest. You can use the Dependency Proxy by adding the URL to your
.gitlab-ci.ymlfile, manually pulling the image from the command line, or using a Dockerfile. Check out the documentation and start saving time on your builds.See Documentation and Issue.