在 Kubernetes (kubeadm) 中使用 Helm 安装 Jenkins
Install Jenkins using Helm in Kubernetes (kubeadm)
首先我在代理环境后面有Kubernetes集群
我有三台服务器，master、node1 和 node2。
我使用以下命令安装了 Jenkins。
创建 jenkine-project 命名空间然后
helm install --name jenkins -f jenkins-values.yaml stable/jenkins --namespace jenkins-project
jenkins-values.yaml 是
Master:
Name: jenkins-master
Image: "jenkins/jenkins"
ImageTag: "lts"
ImagePullPolicy: "Always"
# ImagePullSecret: jenkins
Component: "jenkins-master"
UseSecurity: true
AdminUser: admin
AdminPassword: 1qaz2wsx
Cpu: "200m"
Memory: "512Mi"
# Environment variables that get added to the init container (useful for e.g. http_proxy)
InitContainerEnv:
- name: http_proxy
value: "http://168.219.yyy.zzz:8080"
- name: https_proxy
value: "http://168.219.yyy.zzz:8080"
- name: no_proxy
value: "localhost,127.0.0.1,10.251.141.*,"
ContainerEnv:
- name: http_proxy
value: "http://168.219.yyy.zzz:8080"
- name: https_proxy
value: "http://168.219.yyy.zzz:8080"
JavaOpts: >-
-Dhttp.proxyHost=168.219.yyy.zzz
-Dhttp.proxyPort=8080
-Dhttps.proxyHost=168.219.yyy.zzz
-Dhttps.proxyPort=8080
# Set min/max heap here if needed with:
# JavaOpts: "-Xms512m -Xmx512m"
# JenkinsOpts: ""
# JenkinsUriPrefix: "/jenkins"
# Set RunAsUser to 1000 to let Jenkins run as non-root user 'jenkins' which exists in 'jenkins/jenkins' docker image.
# When setting RunAsUser to a different value than 0 also set FsGroup to the same value:
# RunAsUser: <defaults to 0>
# FsGroup: <will be omitted in deployment if RunAsUser is 0>
ServicePort: 8080
# For minikube, set this to NodePort, elsewhere use LoadBalancer
# Use ClusterIP if your setup includes ingress controller
ServiceType: LoadBalancer
# Master Service annotations
ServiceAnnotations: {}
# service.beta.kubernetes.io/aws-load-balancer-backend-protocol: https
# Used to create Ingress record (should used with ServiceType: ClusterIP)
# HostName: jenkins.cluster.local
# NodePort: <to set explicitly, choose port between 30000-32767
ContainerPort: 8080
# Enable Kubernetes Liveness and Readiness Probes
HealthProbes: true
HealthProbesTimeout: 60
SlaveListenerPort: 50000
# Kubernetes service type for the JNLP slave service
# SETTING THIS TO "LoadBalancer" IS A HUGE SECURITY RISK: https://github.com/kubernetes/charts/issues/1341
SlaveListenerServiceType: ClusterIP
SlaveListenerServiceAnnotations: {}
LoadBalancerSourceRanges:
- 0.0.0.0/0
# Optionally assign a known public LB IP
# LoadBalancerIP: 1.2.3.4
# Optionally configure a JMX port
# requires additional JavaOpts, ie
# JavaOpts: >
# -Dcom.sun.management.jmxremote.port=4000
# -Dcom.sun.management.jmxremote.authenticate=false
# -Dcom.sun.management.jmxremote.ssl=false
# JMXPort: 4000
# List of plugins to be install during Jenkins master start
InstallPlugins:
- kubernetes:1.4
- workflow-aggregator:2.5
- workflow-job:2.17
- credentials-binding:1.16
- p4:1.8.7
- blueocean:1.4.2
# Used to approve a list of groovy functions in pipelines used the script-security plugin. Can be viewed under /scriptApproval
ScriptApproval:
- "method groovy.json.JsonSlurperClassic parseText java.lang.String"
- "new groovy.json.JsonSlurperClassic"
- "staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods leftShift java.util.Map java.util.Map"
- "staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods split java.lang.String"
# List of groovy init scripts to be executed during Jenkins master start
InitScripts:
# - |
# print 'adding global pipeline libraries, register properties, bootstrap jobs...'
# Kubernetes secret that contains a 'credentials.xml' for Jenkins
# CredentialsXmlSecret: jenkins-credentials
# Kubernetes secret that contains files to be put in the Jenkins 'secrets' directory,
# useful to manage encryption keys used for credentials.xml for instance (such as
# master.key and hudson.util.Secret)
# SecretsFilesSecret: jenkins-secrets
# Jenkins XML job configs to provision
# Jobs: |-
# test: |-
# <<xml here>>
CustomConfigMap: false
# Node labels and tolerations for pod assignment
# ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
# ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature
NodeSelector: {}
Tolerations: {}
Ingress:
Annotations:
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
TLS:
# - secretName: jenkins.cluster.local
# hosts:
# - jenkins.cluster.local
Agent:
Enabled: true
Image: jenkins/jnlp-slave
ImageTag: 3.10-1
# ImagePullSecret: jenkins
Component: "jenkins-slave"
Privileged: false
Cpu: "200m"
Memory: "256Mi"
# You may want to change this to true while testing a new image
AlwaysPullImage: false
# You can define the volumes that you want to mount for this container
# Allowed types are: ConfigMap, EmptyDir, HostPath, Nfs, Pod, Secret
# Configure the attributes as they appear in the corresponding Java class for that type
# https://github.com/jenkinsci/kubernetes-plugin/tree/master/src/main/java/org/csanchez/jenkins/plugins/kubernetes/volumes
volumes:
- type: HostPath
secretName: /var/run/docker.sock
mountPath: /var/run/docker.sock
NodeSelector: {}
# Key Value selectors. Ex:
# jenkins-agent: v1
Persistence:
Enabled: true
## A manually managed Persistent Volume and Claim
## Requires Persistence.Enabled: true
## If defined, PVC must be created manually before volume will be bound
# ExistingClaim: pvc-jenkins-master
## jenkins data Persistent Volume Storage Class
## If defined, storageClassName: <storageClass>
## If set to "-", storageClassName: "", which disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, standard on
## GKE, AWS & OpenStack)
##
StorageClass: "jenkins-pv"
Annotations: {}
AccessMode: ReadWriteOnce
Size: 20Gi
volumes:
# - name: nothing
# emptyDir: {}
mounts:
# - mountPath: /var/nothing
# name: nothing
# readOnly: true
NetworkPolicy:
# Enable creation of NetworkPolicy resources.
Enabled: false
# For Kubernetes v1.4, v1.5 and v1.6, use 'extensions/v1beta1'
# For Kubernetes v1.7, use 'networking.k8s.io/v1'
ApiVersion: networking.k8s.io/v1
## Install Default RBAC roles and bindings
rbac:
install: true
serviceAccountName: default
# RBAC api version (currently either v1beta1 or v1alpha1)
apiVersion: v1beta1
# Cluster role reference
roleRef: cluster-admin
并且 pod jenkins-694674f4bd-zqfpq 已创建。
我运行kubectl logs jenkins-694674f4bd-zqfpq -n jenkins-project命令
问题来了。
# kubectl logs jenkins-694674f4bd-zqfpq -n jenkins-project
Error from server: Get https://10.251.141.74:10250/containerLogs/jenkins-project/jenkins-694674f4bd-zqfpq/jenkins: read tcp 10.251.141.xxx:34630->168.219.yyy.zzz:8080: read: connection reset by peer
在此错误消息中，10.251.141.xxx 是主服务器的 IP 地址，
168.219.yyy.zzz:8080为代理地址
而且（我猜）因为这个问题插件将无法正常安装。
这是什么问题，我该如何解决？
据我了解，您在代理后面有一个集群，所以看起来像这样：
You | Proxy| All Kubernetes nodes and master
当您调用 kubectl logs 命令时，kubectl 正在连接到 API 服务器，然后 API 服务器是您 pod 的 getting logs 来自节点。
正如我从命令的输出中看到的，API 服务器正在尝试通过代理服务器而不是直接连接来连接到节点，这就是为什么我认为您的代理设置有点不正确主控上的设置。
尝试在主节点和节点上使用 no_proxy 将所有集群内部 IP 范围添加到异常，我认为它应该有所帮助。
首先我在代理环境后面有Kubernetes集群
我有三台服务器，master、node1 和 node2。
我使用以下命令安装了 Jenkins。
创建 jenkine-project 命名空间然后
helm install --name jenkins -f jenkins-values.yaml stable/jenkins --namespace jenkins-project
jenkins-values.yaml 是
Master:
Name: jenkins-master
Image: "jenkins/jenkins"
ImageTag: "lts"
ImagePullPolicy: "Always"
# ImagePullSecret: jenkins
Component: "jenkins-master"
UseSecurity: true
AdminUser: admin
AdminPassword: 1qaz2wsx
Cpu: "200m"
Memory: "512Mi"
# Environment variables that get added to the init container (useful for e.g. http_proxy)
InitContainerEnv:
- name: http_proxy
value: "http://168.219.yyy.zzz:8080"
- name: https_proxy
value: "http://168.219.yyy.zzz:8080"
- name: no_proxy
value: "localhost,127.0.0.1,10.251.141.*,"
ContainerEnv:
- name: http_proxy
value: "http://168.219.yyy.zzz:8080"
- name: https_proxy
value: "http://168.219.yyy.zzz:8080"
JavaOpts: >-
-Dhttp.proxyHost=168.219.yyy.zzz
-Dhttp.proxyPort=8080
-Dhttps.proxyHost=168.219.yyy.zzz
-Dhttps.proxyPort=8080
# Set min/max heap here if needed with:
# JavaOpts: "-Xms512m -Xmx512m"
# JenkinsOpts: ""
# JenkinsUriPrefix: "/jenkins"
# Set RunAsUser to 1000 to let Jenkins run as non-root user 'jenkins' which exists in 'jenkins/jenkins' docker image.
# When setting RunAsUser to a different value than 0 also set FsGroup to the same value:
# RunAsUser: <defaults to 0>
# FsGroup: <will be omitted in deployment if RunAsUser is 0>
ServicePort: 8080
# For minikube, set this to NodePort, elsewhere use LoadBalancer
# Use ClusterIP if your setup includes ingress controller
ServiceType: LoadBalancer
# Master Service annotations
ServiceAnnotations: {}
# service.beta.kubernetes.io/aws-load-balancer-backend-protocol: https
# Used to create Ingress record (should used with ServiceType: ClusterIP)
# HostName: jenkins.cluster.local
# NodePort: <to set explicitly, choose port between 30000-32767
ContainerPort: 8080
# Enable Kubernetes Liveness and Readiness Probes
HealthProbes: true
HealthProbesTimeout: 60
SlaveListenerPort: 50000
# Kubernetes service type for the JNLP slave service
# SETTING THIS TO "LoadBalancer" IS A HUGE SECURITY RISK: https://github.com/kubernetes/charts/issues/1341
SlaveListenerServiceType: ClusterIP
SlaveListenerServiceAnnotations: {}
LoadBalancerSourceRanges:
- 0.0.0.0/0
# Optionally assign a known public LB IP
# LoadBalancerIP: 1.2.3.4
# Optionally configure a JMX port
# requires additional JavaOpts, ie
# JavaOpts: >
# -Dcom.sun.management.jmxremote.port=4000
# -Dcom.sun.management.jmxremote.authenticate=false
# -Dcom.sun.management.jmxremote.ssl=false
# JMXPort: 4000
# List of plugins to be install during Jenkins master start
InstallPlugins:
- kubernetes:1.4
- workflow-aggregator:2.5
- workflow-job:2.17
- credentials-binding:1.16
- p4:1.8.7
- blueocean:1.4.2
# Used to approve a list of groovy functions in pipelines used the script-security plugin. Can be viewed under /scriptApproval
ScriptApproval:
- "method groovy.json.JsonSlurperClassic parseText java.lang.String"
- "new groovy.json.JsonSlurperClassic"
- "staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods leftShift java.util.Map java.util.Map"
- "staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods split java.lang.String"
# List of groovy init scripts to be executed during Jenkins master start
InitScripts:
# - |
# print 'adding global pipeline libraries, register properties, bootstrap jobs...'
# Kubernetes secret that contains a 'credentials.xml' for Jenkins
# CredentialsXmlSecret: jenkins-credentials
# Kubernetes secret that contains files to be put in the Jenkins 'secrets' directory,
# useful to manage encryption keys used for credentials.xml for instance (such as
# master.key and hudson.util.Secret)
# SecretsFilesSecret: jenkins-secrets
# Jenkins XML job configs to provision
# Jobs: |-
# test: |-
# <<xml here>>
CustomConfigMap: false
# Node labels and tolerations for pod assignment
# ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
# ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature
NodeSelector: {}
Tolerations: {}
Ingress:
Annotations:
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
TLS:
# - secretName: jenkins.cluster.local
# hosts:
# - jenkins.cluster.local
Agent:
Enabled: true
Image: jenkins/jnlp-slave
ImageTag: 3.10-1
# ImagePullSecret: jenkins
Component: "jenkins-slave"
Privileged: false
Cpu: "200m"
Memory: "256Mi"
# You may want to change this to true while testing a new image
AlwaysPullImage: false
# You can define the volumes that you want to mount for this container
# Allowed types are: ConfigMap, EmptyDir, HostPath, Nfs, Pod, Secret
# Configure the attributes as they appear in the corresponding Java class for that type
# https://github.com/jenkinsci/kubernetes-plugin/tree/master/src/main/java/org/csanchez/jenkins/plugins/kubernetes/volumes
volumes:
- type: HostPath
secretName: /var/run/docker.sock
mountPath: /var/run/docker.sock
NodeSelector: {}
# Key Value selectors. Ex:
# jenkins-agent: v1
Persistence:
Enabled: true
## A manually managed Persistent Volume and Claim
## Requires Persistence.Enabled: true
## If defined, PVC must be created manually before volume will be bound
# ExistingClaim: pvc-jenkins-master
## jenkins data Persistent Volume Storage Class
## If defined, storageClassName: <storageClass>
## If set to "-", storageClassName: "", which disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, standard on
## GKE, AWS & OpenStack)
##
StorageClass: "jenkins-pv"
Annotations: {}
AccessMode: ReadWriteOnce
Size: 20Gi
volumes:
# - name: nothing
# emptyDir: {}
mounts:
# - mountPath: /var/nothing
# name: nothing
# readOnly: true
NetworkPolicy:
# Enable creation of NetworkPolicy resources.
Enabled: false
# For Kubernetes v1.4, v1.5 and v1.6, use 'extensions/v1beta1'
# For Kubernetes v1.7, use 'networking.k8s.io/v1'
ApiVersion: networking.k8s.io/v1
## Install Default RBAC roles and bindings
rbac:
install: true
serviceAccountName: default
# RBAC api version (currently either v1beta1 or v1alpha1)
apiVersion: v1beta1
# Cluster role reference
roleRef: cluster-admin
并且 pod jenkins-694674f4bd-zqfpq 已创建。
我运行kubectl logs jenkins-694674f4bd-zqfpq -n jenkins-project命令
问题来了。
# kubectl logs jenkins-694674f4bd-zqfpq -n jenkins-project
Error from server: Get https://10.251.141.74:10250/containerLogs/jenkins-project/jenkins-694674f4bd-zqfpq/jenkins: read tcp 10.251.141.xxx:34630->168.219.yyy.zzz:8080: read: connection reset by peer
在此错误消息中，10.251.141.xxx 是主服务器的 IP 地址， 168.219.yyy.zzz:8080为代理地址
而且（我猜）因为这个问题插件将无法正常安装。
这是什么问题，我该如何解决？
据我了解，您在代理后面有一个集群，所以看起来像这样：
You | Proxy| All Kubernetes nodes and master
当您调用 kubectl logs 命令时，kubectl 正在连接到 API 服务器，然后 API 服务器是您 pod 的 getting logs 来自节点。
正如我从命令的输出中看到的，API 服务器正在尝试通过代理服务器而不是直接连接来连接到节点，这就是为什么我认为您的代理设置有点不正确主控上的设置。
尝试在主节点和节点上使用 no_proxy 将所有集群内部 IP 范围添加到异常，我认为它应该有所帮助。