如何在 Kubernetes 上使用 Docker 运行 Jenkins？

Question

我正在尝试在 Kubernetes 上执行 Jenkins & Docker CLI 容器。这是我的步骤：

我使用以下方法创建广告连播：

kubectl --kubeconfig my-kubeconfig.yml run my-jenkins-pod --image=trion/jenkins-docker-client --restart=Never

根据图像创建一个 pod-https://hub.docker.com/r/trion/jenkins-docker-client

我使用以下方法创建部署：

kubectl --kubeconfig my-kubeconfig.yml apply -f /kub/kube

/kub/kube 包含 jenkins-deployment-yaml 我已将其配置为：

apiVersion: v1
kind: Service
metadata:
  name: my-jenkins-pod
spec:
  ports:
    - protocol: "TCP"
      port: 50000
      targetPort: 5001
  selector:
    app: my-jenkins-pod
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-jenkins-pod
spec:
  selector:
    matchLabels:
      app: my-jenkins-pod
  replicas: 1
  template:
    metadata:
      labels:
        app: my-jenkins-pod
    spec:
      containers:
        - name: ml-services
          image: trion/jenkins-docker-client
          ports:
            - containerPort: 5001

要访问 Jenkins 容器，我使用以下方法公开 IP：

kubectl --kubeconfig my-kubeconfig.yml expose deployment my-jenkins-pod --type=LoadBalancer --name=my-jenkins-pod-public

到 return Jenkins 的 IP 和 Docker 我使用的图像 :

kubectl --kubeconfig my-kubeconfig.yml get services my-jenkins-pod-public

哪个 returns:

NAME                    TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)          AGE
my-jenkins-pod-public   LoadBalancer   9.5.52.28   161.61.222.16   5001:30878/TCP   10m

为了测试，我在以下位置打开 URL：

http://161.61.222.16:5001/

哪个 returns:

This page isn’t working
161.61.222.16 didn’t send any data.
ERR_EMPTY_RESPONSE

好像服务已经启动但是端口映射不正确？

pod my-jenkins-pod 的日志包含：

Running from: /usr/share/jenkins/jenkins.war webroot: EnvVars.masterEnvVars.get("JENKINS_HOME") 2021-04-03 11:15:42.899+0000 [id=1] INFO org.eclipse.jetty.util.log.Log#initialized: Logging initialized @274ms to org.eclipse.jetty.util.log.JavaUtilLog 2021-04-03 11:15:43.012+0000 [id=1] INFO winstone.Logger#logInternal: Beginning extraction from war file 2021-04-03 11:15:44.369+0000 [id=1] WARNING o.e.j.s.handler.ContextHandler#setContextPath: Empty contextPath 2021-04-03 11:15:44.416+0000 [id=1] INFO org.eclipse.jetty.server.Server#doStart: jetty-9.4.39.v20210325; built: 2021-03-25T14:42:11.471Z; git: 9fc7ca5a922f2a37b84ec9dbc26a5168cee7e667; jvm 1.8.0_282-b08 2021-04-03 11:15:44.653+0000 [id=1] INFO o.e.j.w.StandardDescriptorProcessor#visitServlet: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet 2021-04-03 11:15:44.695+0000 [id=1] INFO o.e.j.s.s.DefaultSessionIdManager#doStart: DefaultSessionIdManager workerName=node0 2021-04-03 11:15:44.695+0000 [id=1] INFO o.e.j.s.s.DefaultSessionIdManager#doStart: No SessionScavenger set, using defaults 2021-04-03 11:15:44.696+0000 [id=1] INFO o.e.j.server.session.HouseKeeper#startScavenging: node0 Scavenging every 660000ms 2021-04-03 11:15:45.081+0000 [id=1] INFO hudson.WebAppMain#contextInitialized: Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME") 2021-04-03 11:15:45.203+0000 [id=1] INFO o.e.j.s.handler.ContextHandler#doStart: Started w.@24f43aa3{Jenkins v2.286,/,file:///var/jenkins_home/war/,AVAILABLE}{/var/jenkins_home/war} 2021-04-03 11:15:45.241+0000 [id=1] INFO o.e.j.server.AbstractConnector#doStart: Started ServerConnector@4f0f2942{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} 2021-04-03 11:15:45.241+0000 [id=1] INFO org.eclipse.jetty.server.Server#doStart: Started @2616ms 2021-04-03 11:15:45.245+0000 [id=21] INFO winstone.Logger#logInternal: Winstone Servlet Engine running: controlPort=disabled 2021-04-03 11:15:46.479+0000 [id=26] INFO jenkins.InitReactorRunner#onAttained: Started initialization 2021-04-03 11:15:46.507+0000 [id=26] INFO jenkins.InitReactorRunner#onAttained: Listed all plugins 2021-04-03 11:15:47.654+0000 [id=27] INFO jenkins.InitReactorRunner#onAttained: Prepared all plugins 2021-04-03 11:15:47.660+0000 [id=26] INFO jenkins.InitReactorRunner#onAttained: Started all plugins 2021-04-03 11:15:47.680+0000 [id=27] INFO jenkins.InitReactorRunner#onAttained: Augmented all extensions 2021-04-03 11:15:48.620+0000 [id=26] INFO jenkins.InitReactorRunner#onAttained: System config loaded 2021-04-03 11:15:48.621+0000 [id=26] INFO jenkins.InitReactorRunner#onAttained: System config adapted 2021-04-03 11:15:48.621+0000 [id=27] INFO jenkins.InitReactorRunner#onAttained: Loaded all jobs 2021-04-03 11:15:48.622+0000 [id=27] INFO jenkins.InitReactorRunner#onAttained: Configuration for all jobs updated 2021-04-03 11:15:48.704+0000 [id=40] INFO hudson.model.AsyncPeriodicWork#lambda$doRun[=32=]: Started Download metadata 2021-04-03 11:15:48.722+0000 [id=40] INFO hudson.util.Retrier#start: Attempt #1 to do the action check updates server 2021-04-03 11:15:49.340+0000 [id=26] INFO jenkins.install.SetupWizard#init:

************************************************************* Jenkins initial setup is required. An admin user has been created and a password generated. Please use the following password to proceed to installation: ab5dbf74145c405fb5a33456d4b97436 This may also be found at: /var/jenkins_home/secrets/initialAdminPassword

************************************************************* 2021-04-03 11:16:08.107+0000 [id=27] INFO jenkins.InitReactorRunner#onAttained: Completed initialization 2021-04-03 11:16:08.115+0000 [id=20] INFO hudson.WebAppMain#run: Jenkins is fully up and running 2021-04-03 11:16:08.331+0000 [id=40] INFO h.m.DownloadService$Downloadable#load: Obtained the updated data file for hudson.tasks.Maven.MavenInstaller 2021-04-03 11:16:08.332+0000 [id=40] INFO hudson.util.Retrier#start: Performed the action check updates server successfully at the attempt #1 2021-04-03 11:16:08.334+0000 [id=40] INFO hudson.model.AsyncPeriodicWork#lambda$doRun[=32=]: Finished Download metadata. 19,626 ms

Jenkins 服务器是否在 8080 端口启动？由于此日志消息：

11:15:45.241+0000 [id=1] INFO o.e.j.server.AbstractConnector#doStart: Started ServerConnector@4f0f2942{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} 2021-04-03

我尝试将 jenkins-deployment-yaml 更改为指向端口 8080 而不是 50000，导致更新后的 jenkins-deployment-yaml :

apiVersion: v1
kind: Service
metadata:
  name: my-jenkins-pod
spec:
  ports:
    - protocol: "TCP"
      port: 8080

但是当我尝试访问 http://161.61.222.16:5001/

时 return 出现了同样的错误

我的端口映射不正确吗？这是将 docker 集线器上可用的现有 docker 容器添加到 Kubernetes 集群的正确方法吗？

更新：

命令kubectl describe services my-jenkins-pod-public的结果是：

Name:                     my-jenkins-pod-public
Namespace:                default
Labels:                   <none>
Annotations:              kubernetes.digitalocean.com/load-balancer-id: d46ae9ae-6e8a-4fd8-aa58-43c08310059a
Selector:                 app=my-jenkins-pod
Type:                     LoadBalancer
IP Families:              <none>
IP:                       10.245.152.228
IPs:                      10.245.152.228
LoadBalancer Ingress:     161.61.222.16
Port:                     <unset>  5001/TCP
TargetPort:               5001/TCP
NodePort:                 <unset>  30878/TCP
Endpoints:                10.214.12.12:5001
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

正在尝试通过浏览器访问 http://161.61.222.16:30878/ returns:

This site can’t be reached159.65.211.46 refused to connect. Try:

Checking the connection Checking the proxy and the firewall ERR_CONNECTION_REFUSED

正在尝试通过浏览器访问 http://161.61.222.16:5001/ returns:

This page isn’t working 161.61.222.16 didn’t send any data. ERR_EMPTY_RESPONSE

似乎端口 5001 是 exposed/accessible 但未发送任何数据。

我也尝试在端口 5001 和 30878 上访问 10.214.12.12 但两个请求都超时。

Answer 1

您需要从运行正在运行容器的主机外部使用 http://161.61.222.16:30878/。端口 5001 只能在具有内部 IP 的集群内部访问（在您的情况下为 9.5.52.28）。每当您公开您的部署时，自动（您也可以手动定义）NodePort 之一（默认在 30000 - 32767 之间）分配给外部请求的服务。

有关服务详情，您需要运行以下命令。命令输出将为您提供 NodePort 和其他详细信息。

kubectl describe services my-service

请查看相关kubernetes documentation

您还使用端口 5001 配置服务，但据我在日志中看到的，Jenkins 使用 8080。尝试将目标服务端口从 5001 更改为 8080

如何在 Kubernetes 上使用 Docker 运行 Jenkins？

How to run Jenkins with Docker on Kubernetes?

jenkins

docker

digital-ocean

kubernetes

jenkins-pipeline