gitlab-runner 在查询工作时偶尔会得到 403 "Access Forbidden"

Question

我有两个 Ubuntu 16.04.5 LTS 服务器。一个是运行 gitlab-ee 实例，另一个是运行 gitlab-runners。

当我将代码推送到服务器时，我注意到我的共享运行器需要很长时间才能获取代码并构建它。

我查看了 /var/log/gitlab/gitlab-rails/api_json.log 下的 gitlab-ee 日志，发现它们经常出现 403 错误。

{"time":"2018-09-03T17:58:29.432Z","severity":"INFO","duration":5.41,"db":1.34,"view":4.07,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":{"info":{"name":"gitlab-runner","version":"11.2.0","revision":"35e8515d","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null}},"token":"[FILTERED]","last_update":"c565c8f1c839e48b27a1758c04af7863"},"host":"gitlab.XXXX.XXX","ip":"XX.XX.XX.XX","ua":"gitlab-runner 11.2.0 (11-2-stable; go1.8.7; linux/amd64)","queue_duration":8.48}
{"time":"2018-09-03T17:58:29.621Z","severity":"INFO","duration":5.51,"db":1.26,"view":4.25,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":{"info":{"name":"gitlab-runner","version":"11.2.0","revision":"35e8515d","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null}},"token":"[FILTERED]","last_update":"6c328f52ff65c51b4b34b9c1ea26249e"},"host":"gitlab.XXXX.XXX","ip":"XX.XX.XX.XX","ua":"gitlab-runner 11.2.0 (11-2-stable; go1.8.7; linux/amd64)","queue_duration":9.43}
{"time":"2018-09-03T17:58:29.807Z","severity":"INFO","duration":5.5,"db":1.61,"view":3.8899999999999997,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":{"info":{"name":"gitlab-runner","version":"11.2.0","revision":"35e8515d","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null}},"token":"[FILTERED]","last_update":"7d3fda493909db2329c6a578ad9960ec"},"host":"gitlab.XXXX.XXX","ip":"XX.XX.XX.XX","ua":"gitlab-runner 11.2.0 (11-2-stable; go1.8.7; linux/amd64)","queue_duration":7.72}

直到，每隔一段时间，一个人就成功了，

{"time":"2018-09-03T19:22:07.249Z","severity":"INFO","duration":24.36,"db":7.55,"view":16.81,"status":204,"method":"POST","path":"/api/v4/jobs/request","params":{"info":{"name":"gitlab-runner","version":"11.2.0","revision":"35e8515d","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null}},"token":"[FILTERED]","last_update":"e0d8576707ef9261fd3e59106f8a2ba8"},"host":"gitlab.XXXX.XXX","ip":"XX.XX.XX.XX","ua":"gitlab-runner 11.2.0 (11-2-stable; go1.8.7; linux/amd64)","queue_duration":18.47}

这会导致排队时间超过 10 分钟。

我试图找出造成这种情况的原因，但无法找到。我采取的步骤是：

删除所有跑步者并重新创建它们。
验证跑步者，没有任何问题

好像很像，但我没有额外安装任何东西。这是一个普通的 gitlab-ee 实例。

Answer 1

你的 GitLab 实例是否在负载均衡器后面？过去，我的自托管 GitLab EE 实例遇到过非常相似的情况。由于负载均衡器，GitLab 看到所有请求都来自同一个 IP 地址，并且会一直错误地发出临时禁令。我在 GitLab Runner 作业请求等方面遇到了 403 响应。

为了修复我的安装，我最终完全关闭了机架攻击过滤。不过，也有一种方法可以转发实际的客户端 IP。

gitlab-runner 在查询工作时偶尔会得到 403 "Access Forbidden"

gitlab-runner gets occasionally 403 "Access Forbidden" while querying for jobs

gitlab

gitlab-ci-runner