错误 运行 google-gitlab 上的 cloud-sdk CI
Error running google-cloud-sdk on gitlab CI
我正在尝试将 google 数据存储模拟器 运行 作为 CI 的一部分。我已将 services 部分添加到 .gitlab-ci.yml，但出现错误。
这是完整的配置 YAML：
image: python:3.9-slim
# Change pip's cache directory to be inside the project directory since we can
# only cache local items.
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
# Pip's cache doesn't store the python packages
# https://pip.pypa.io/en/stable/reference/pip_install/#caching
#
# If you want to also cache the installed packages, you have to install
# them in a virtualenv and cache it as well.
cache:
paths:
- .cache/pip
variables:
DATASTORE_DATASET: project-007
DATASTORE_EMULATOR_HOST: google-cloud-sdk:8081
DATASTORE_EMULATOR_HOST_PATH: google-cloud-sdk:8081/datastore
DATASTORE_HOST: http://google-cloud-sdk:8081
DATASTORE_PROJECT_ID: project-007
before_script:
- apt-get update && apt-get install -y ffmpeg make
- python --version # Print out python version for debugging
- cd backend
- python -m pip install -r dev-requirements.txt
test:
services:
- name: google/cloud-sdk:latest
command: ["gcloud", "beta", "emulators", "datastore", "start", "--host-port", "0.0.0.0:8080"]
variables:
CLOUDSDK_CORE_PROJECT: project-007
script:
- make test
我在 CI 日志中看到的错误是：
*** WARNING: Service runner--azerasq-project-26594908-concurrent-0-aababf5d82e87691-google__cloud-sdk-0 probably didn't start properly.
Health check error:
service "runner--azerasq-project-26594908-concurrent-0-aababf5d82e87691-google__cloud-sdk-0-wait-for-service" health check: exit code 1
Health check container logs:
2021-11-25T15:49:31.057994970Z FATAL: No HOST or PORT found
Service container logs:
2021-11-25T15:49:30.783697828Z Executing: /usr/lib/google-cloud-sdk/platform/cloud-datastore-emulator/cloud_datastore_emulator create --project_id=gitlab-ci-plan-free-5-6ef84d /root/.config/gcloud/emulators/datastore
*********
关于如何解决此问题或 运行 gitlab CI 中的数据存储模拟器的更好方法有什么想法吗？
谢谢！
当您 运行 docker run -it -e CLOUDSDK_CORE_PROJECT=test google/cloud-sdk:latest gcloud beta emulators datastore start --host-port 0.0.0.0:8080 时模拟器 运行 正确，这不是您的容器命令的问题。这意味着这是 GitLab 的 运行ner 设置中的问题，或者是您使用 运行ner 设置的方式中的问题。 Specifically, how the services health check works.
您看到的问题是 google/cloud-sdk 图像没有暴露端口，因为它是一个 CLI（而不是 Web 应用程序）并且暴露端口的行为是模拟器。这意味着服务健康检查无法找到检查容器健康的端口，并立即失败。要解决这个问题，只需基于SDK构建自己的镜像，并暴露8080端口即可：
FROM google/cloud-sdk
EXPOSE 8080
然后更新您的服务块以改用该容器：
services:
- name: $CI_REGISTRY_IMAGE:latest
alias: google-test
command: ["gcloud", "beta", "emulators", "datastore", "start", "--host-port", "0.0.0.0:8080"]
variables:
CLOUDSDK_CORE_PROJECT: project-007
您的服务现在将适当地通过健康检查。这是我在 gitlab 仓库中测试的输出：
Running with gitlab-runner 14.4.0-rc1 (bc99a056)
on blue-5.shared.runners-manager.gitlab.com/default -AzERasQ
Preparing the "docker+machine" executor 01:26
Using Docker executor with image alpine:latest ...
Starting service registry.gitlab.com/patrick/service-test:latest ...
Authenticating with credentials from job payload (GitLab Registry)
Pulling docker image registry.gitlab.com/patrick/service-test:latest ...
Using docker image sha256:954ea7f75a0a88b281d46cbaa1c359e0782268a6c5390a722ebea387893e7a47 for registry.gitlab.com/patrick/service-test:latest with digest registry.gitlab.com/patrick/service-test@sha256:8d240acba0977cc8deea217026d23678e471a5453ffd4dbeeb4197aa1b6e024d ...
Waiting for services to be up and running...
Pulling docker image alpine:latest ...
Using docker image sha256:c059bfaa849c4d8e4aecaeb3a10c2d9b3d85f5165c66ad3a4d937758128c4d18 for alpine:latest with digest alpine@sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300 ...
Preparing environment 00:01
Running on runner--azerasq-project-31673063-concurrent-0 via runner-azerasq-shared-1638120050-4cf20e79...
Getting source from Git repository 00:02
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/patrick/service-test/.git/
Created fresh repository.
Checking out a4825851 as main...
Skipping Git submodules setup
Executing "step_script" stage of the job script
Using docker image sha256:c059bfaa849c4d8e4aecaeb3a10c2d9b3d85f5165c66ad3a4d937758128c4d18 for alpine:latest with digest alpine@sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300 ...
$ ping google-test
PING google-test (172.17.0.3): 56 data bytes
64 bytes from 172.17.0.3: seq=0 ttl=64 time=0.070 ms
64 bytes from 172.17.0.3: seq=1 ttl=64 time=0.066 ms
我正在尝试将 google 数据存储模拟器 运行 作为 CI 的一部分。我已将 services 部分添加到 .gitlab-ci.yml，但出现错误。
这是完整的配置 YAML：
image: python:3.9-slim
# Change pip's cache directory to be inside the project directory since we can
# only cache local items.
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
# Pip's cache doesn't store the python packages
# https://pip.pypa.io/en/stable/reference/pip_install/#caching
#
# If you want to also cache the installed packages, you have to install
# them in a virtualenv and cache it as well.
cache:
paths:
- .cache/pip
variables:
DATASTORE_DATASET: project-007
DATASTORE_EMULATOR_HOST: google-cloud-sdk:8081
DATASTORE_EMULATOR_HOST_PATH: google-cloud-sdk:8081/datastore
DATASTORE_HOST: http://google-cloud-sdk:8081
DATASTORE_PROJECT_ID: project-007
before_script:
- apt-get update && apt-get install -y ffmpeg make
- python --version # Print out python version for debugging
- cd backend
- python -m pip install -r dev-requirements.txt
test:
services:
- name: google/cloud-sdk:latest
command: ["gcloud", "beta", "emulators", "datastore", "start", "--host-port", "0.0.0.0:8080"]
variables:
CLOUDSDK_CORE_PROJECT: project-007
script:
- make test
我在 CI 日志中看到的错误是：
*** WARNING: Service runner--azerasq-project-26594908-concurrent-0-aababf5d82e87691-google__cloud-sdk-0 probably didn't start properly.
Health check error:
service "runner--azerasq-project-26594908-concurrent-0-aababf5d82e87691-google__cloud-sdk-0-wait-for-service" health check: exit code 1
Health check container logs:
2021-11-25T15:49:31.057994970Z FATAL: No HOST or PORT found
Service container logs:
2021-11-25T15:49:30.783697828Z Executing: /usr/lib/google-cloud-sdk/platform/cloud-datastore-emulator/cloud_datastore_emulator create --project_id=gitlab-ci-plan-free-5-6ef84d /root/.config/gcloud/emulators/datastore
*********
关于如何解决此问题或 运行 gitlab CI 中的数据存储模拟器的更好方法有什么想法吗？ 谢谢！
当您 运行 docker run -it -e CLOUDSDK_CORE_PROJECT=test google/cloud-sdk:latest gcloud beta emulators datastore start --host-port 0.0.0.0:8080 时模拟器 运行 正确，这不是您的容器命令的问题。这意味着这是 GitLab 的 运行ner 设置中的问题，或者是您使用 运行ner 设置的方式中的问题。 Specifically, how the services health check works.
您看到的问题是 google/cloud-sdk 图像没有暴露端口，因为它是一个 CLI（而不是 Web 应用程序）并且暴露端口的行为是模拟器。这意味着服务健康检查无法找到检查容器健康的端口，并立即失败。要解决这个问题，只需基于SDK构建自己的镜像，并暴露8080端口即可：
FROM google/cloud-sdk
EXPOSE 8080
然后更新您的服务块以改用该容器：
services:
- name: $CI_REGISTRY_IMAGE:latest
alias: google-test
command: ["gcloud", "beta", "emulators", "datastore", "start", "--host-port", "0.0.0.0:8080"]
variables:
CLOUDSDK_CORE_PROJECT: project-007
您的服务现在将适当地通过健康检查。这是我在 gitlab 仓库中测试的输出：
Running with gitlab-runner 14.4.0-rc1 (bc99a056)
on blue-5.shared.runners-manager.gitlab.com/default -AzERasQ
Preparing the "docker+machine" executor 01:26
Using Docker executor with image alpine:latest ...
Starting service registry.gitlab.com/patrick/service-test:latest ...
Authenticating with credentials from job payload (GitLab Registry)
Pulling docker image registry.gitlab.com/patrick/service-test:latest ...
Using docker image sha256:954ea7f75a0a88b281d46cbaa1c359e0782268a6c5390a722ebea387893e7a47 for registry.gitlab.com/patrick/service-test:latest with digest registry.gitlab.com/patrick/service-test@sha256:8d240acba0977cc8deea217026d23678e471a5453ffd4dbeeb4197aa1b6e024d ...
Waiting for services to be up and running...
Pulling docker image alpine:latest ...
Using docker image sha256:c059bfaa849c4d8e4aecaeb3a10c2d9b3d85f5165c66ad3a4d937758128c4d18 for alpine:latest with digest alpine@sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300 ...
Preparing environment 00:01
Running on runner--azerasq-project-31673063-concurrent-0 via runner-azerasq-shared-1638120050-4cf20e79...
Getting source from Git repository 00:02
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/patrick/service-test/.git/
Created fresh repository.
Checking out a4825851 as main...
Skipping Git submodules setup
Executing "step_script" stage of the job script
Using docker image sha256:c059bfaa849c4d8e4aecaeb3a10c2d9b3d85f5165c66ad3a4d937758128c4d18 for alpine:latest with digest alpine@sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300 ...
$ ping google-test
PING google-test (172.17.0.3): 56 data bytes
64 bytes from 172.17.0.3: seq=0 ttl=64 time=0.070 ms
64 bytes from 172.17.0.3: seq=1 ttl=64 time=0.066 ms