运行 时 Kubectl 不断重启容器
Kubectl keeps restarting container when ran
我目前正在尝试将我们公司的 squid 服务器迁移到 dockerized 版本，我正在努力让它与 Kubernetes 一起工作。
我构建了一个 Docker 图像，在 运行 和 "docker run" 时工作得非常好。
完整的 Docker 运行 命令是：
sudo docker run -d -i -t --privileged --volume=/proc/sys/net/ipv4/ip_nonlocal_bind:/var/proc/sys/net/ipv4/ip_nonlocal_bind --net=host --cap-add=SYS_MODULE --cap-add=NET_ADMIN --cap-add=NET_RAW -v /dev:/dev -v /lib/modules:/lib/modules -p80:80 -p8080:8080 -p53:53/udp -p5353:5353/udp -p5666:5666/udp -p4500:4500/udp -p500:500/udp -p3306:3306 --name=edge crossense/edge:latest /bin/bash
当我尝试使用 Kubernetes 运行 图像时，使用类似的东西：
kubectl run --image=crossense/edge:latest --port=80 --port=8080 --port=53 --port=5353 --port=5666 --port=4500 --port=500 --port=3306 edge
似乎 Kubernetes 试图启动容器并 运行ning，但没有任何成功...
$kubectl get po
NAME READY REASON RESTARTS AGE
edge-sz7wp 0/1 Running 10 15m
$kubectl describe pod edge 命令给了我很多这样的东西：
Thu, 09 Nov 2017 17:13:05 +0000 Thu, 09 Nov 2017 17:13:05 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id abcc2ff25a624a998871e02bcb62d42d6f39e9db0a39f601efa4d357dd8334aa
Thu, 09 Nov 2017 17:13:15 +0000 Thu, 09 Nov 2017 17:13:15 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id 878778836bd3cc25bdf1e3b9cc2f2f6fa22b75b938a481172f08a6ec50571582
Thu, 09 Nov 2017 17:13:15 +0000 Thu, 09 Nov 2017 17:13:15 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id 878778836bd3cc25bdf1e3b9cc2f2f6fa22b75b938a481172f08a6ec50571582
Thu, 09 Nov 2017 17:13:25 +0000 Thu, 09 Nov 2017 17:13:25 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id aa51e94536216b905ff9ba07951fedbc0007476b55dfdb2e5106418fb6aee05c
Thu, 09 Nov 2017 17:13:25 +0000 Thu, 09 Nov 2017 17:13:25 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id aa51e94536216b905ff9ba07951fedbc0007476b55dfdb2e5106418fb6aee05c
Thu, 09 Nov 2017 17:13:35 +0000 Thu, 09 Nov 2017 17:13:35 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id f4661e5ea33471cd1ba30816b40c8ba2d204fa22509b973da4af6eedb64c592e
Thu, 09 Nov 2017 17:13:35 +0000 Thu, 09 Nov 2017 17:13:35 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id f4661e5ea33471cd1ba30816b40c8ba2d204fa22509b973da4af6eedb64c592e
Thu, 09 Nov 2017 17:13:45 +0000 Thu, 09 Nov 2017 17:13:45 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id 75f83dcb9b4f8af5134d6fd2edcd9342ecf56111e132a45f4e9787e83466e28b
Thu, 09 Nov 2017 17:13:45 +0000 Thu, 09 Nov 2017 17:13:45 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id 75f83dcb9b4f8af5134d6fd2edcd9342ecf56111e132a45f4e9787e83466e28b
Thu, 09 Nov 2017 17:13:55 +0000 Thu, 09 Nov 2017 17:13:55 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id c9d0535b3962ec9da29c068dbb0a6b64426a5ac3e52f72e79bcbaf03c9f3d403
Thu, 09 Nov 2017 17:13:55 +0000 Thu, 09 Nov 2017 17:13:55 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id c9d0535b3962ec9da29c068dbb0a6b64426a5ac3e52f72e79bcbaf03c9f3d403
Thu, 09 Nov 2017 17:14:05 +0000 Thu, 09 Nov 2017 17:14:05 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id 579f4428e9804404bd746cceee88bb6c73066a33263202bb5f1eb15f6ff26d7b
Thu, 09 Nov 2017 17:14:05 +0000 Thu, 09 Nov 2017 17:14:05 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id 579f4428e9804404bd746cceee88bb6c73066a33263202bb5f1eb15f6ff26d7b
Thu, 09 Nov 2017 17:14:15 +0000 Thu, 09 Nov 2017 17:14:15 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id d36b2c9ddf0b1a05d86b43d2a92eb3c00ae92d00e155d5a1be1da8e2682f901b
Thu, 09 Nov 2017 17:14:15 +0000 Thu, 09 Nov 2017 17:14:15 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id d36b2c9ddf0b1a05d86b43d2a92eb3c00ae92d00e155d5a1be1da8e2682f901b
Thu, 09 Nov 2017 17:14:25 +0000 Thu, 09 Nov 2017 17:14:25 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id 2d7b24537414f5e6f2981bf5f01596b19ea1abdb0eb4b81508fc7f44e8c34609
Thu, 09 Nov 2017 17:14:25 +0000 Thu, 09 Nov 2017 17:14:25 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id 2d7b24537414f5e6f2981bf5f01596b19ea1abdb0eb4b81508fc7f44e8c34609
Thu, 09 Nov 2017 17:14:35 +0000 Thu, 09 Nov 2017 17:14:35 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id fdae44c599b77d44839e4897b750203c183001a6053c926432ef5a3c7f4deb38
Thu, 09 Nov 2017 17:14:35 +0000 Thu, 09 Nov 2017 17:14:35 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id fdae44c599b77d44839e4897b750203c183001a6053c926432ef5a3c7f4deb38
Thu, 09 Nov 2017 17:14:45 +0000 Thu, 09 Nov 2017 17:14:45 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id 544351dda838d698e3bc125840edb6ad71cd0165a970cce46825df03b826eb38
Thu, 09 Nov 2017 17:14:45 +0000 Thu, 09 Nov 2017 17:14:45 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id 544351dda838d698e3bc125840edb6ad71cd0165a970cce46825df03b826eb38
Thu, 09 Nov 2017 17:14:55 +0000 Thu, 09 Nov 2017 17:14:55 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id 00fe4c286c1cc9b905c9c0927f82b39d45d41295a9dd0852131bba087bb19610
Thu, 09 Nov 2017 17:14:55 +0000 Thu, 09 Nov 2017 17:14:55 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id 00fe4c286c1cc9b905c9c0927f82b39d45d41295a9dd0852131bba087bb19610
如有任何帮助，我们将不胜感激！
虽然我不能在没有重新生成能力和缺少日志的情况下得出结论，但很容易注意到的区别之一是您在 docker 命令中提供的权限，例如 NET_ADMIN 或 NET_RAW 等在 Kubernetes 运行 命令中缺失。
Kubernetes 还提供了在 pod 声明的 securityContext 中将此类权限分配给具有 capabilities 的 pod 的能力。
我不确定您是否可以使用 Kubectl 执行此操作，但如果您使用 Pod 的 YAML 声明，规范大致如下：
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: myshell
image: "ubuntu:14.04"
command:
- /bin/sleep
- "300"
securityContext:
capabilities:
add:
- NET_ADMIN
如需更多参考，我建议快速浏览一下：
致所有找不到答案的可怜人，
pod一直重启的原因是它执行的命令已经退出，代码为0（表示成功）。
在我的例子中，我是 运行 /bin/bash 作为入口点命令，在我的 pod 配置 .yaml 文件中指定：
apiVersion: v1
kind: Pod
metadata:
name: edge
spec:
containers:
- name: edge
image: "crossense/edge:production"
command:
- /bin/bash
securityContext:
capabilities:
add:
- NET_ADMIN
- SYS_MODULE
- NET_RAW
volumeMounts:
- name: ip-nonlocal-bind
mountPath: /host/proc/sys/net/ipv4
- name: dev
mountPath: /host/dev
- name: modules
mountPath: /host/lib/modules
....
The solution was simply adding a non exiting command to the
entrypoint. This can be any process run on foreground or simply a
/bin/sleep
为了示例和以后的学习，我最终的 pod 配置文件如下所示：
apiVersion: v1
kind: Pod
metadata:
name: edge
spec:
hostNetwork: true
containers:
- name: edge
image: "crossense/edge:production"
command: ["/bin/bash", "-c"]
args: ["service rsyslog restart; service proxysql start; service mongodb start; service pdns-recursor start; service supervisor start; service danted start; touch /var/run/squid.pid; chown proxy /var/run/squid.pid; service squid restart; service ipsec start; /sbin/iptables-restore < /etc/iptables/rules.v4; sleep infinity"]
securityContext:
privileged: true
capabilities:
add:
- NET_ADMIN
- SYS_MODULE
- NET_RAW
volumeMounts:
- mountPath: /dev/shm
name: dshm
- name: ip-nonlocal-bind
mountPath: /host/proc/sys/net/ipv4
- name: dev
mountPath: /dev
- name: modules
mountPath: /lib/modules
ports:
- containerPort: 80
- containerPort: 8080
- containerPort: 53
protocol: UDP
- containerPort: 5353
protocol: UDP
- containerPort: 5666
- containerPort: 4500
- containerPort: 500
- containerPort: 3306
volumes:
- name: dshm
emptyDir:
medium: Memory
- name: ip-nonlocal-bind
hostPath:
path: /proc/sys/net/ipv4
- name: dev
hostPath:
path: /dev
type: Directory
- name: modules
hostPath:
path: /lib/modules
type: Directory
如有任何问题，请随时对此主题发表评论，或通过 max.vlashchuk@gmail.com 询问我 :)
我目前正在尝试将我们公司的 squid 服务器迁移到 dockerized 版本，我正在努力让它与 Kubernetes 一起工作。
我构建了一个 Docker 图像，在 运行 和 "docker run" 时工作得非常好。 完整的 Docker 运行 命令是：
sudo docker run -d -i -t --privileged --volume=/proc/sys/net/ipv4/ip_nonlocal_bind:/var/proc/sys/net/ipv4/ip_nonlocal_bind --net=host --cap-add=SYS_MODULE --cap-add=NET_ADMIN --cap-add=NET_RAW -v /dev:/dev -v /lib/modules:/lib/modules -p80:80 -p8080:8080 -p53:53/udp -p5353:5353/udp -p5666:5666/udp -p4500:4500/udp -p500:500/udp -p3306:3306 --name=edge crossense/edge:latest /bin/bash
当我尝试使用 Kubernetes 运行 图像时，使用类似的东西：
kubectl run --image=crossense/edge:latest --port=80 --port=8080 --port=53 --port=5353 --port=5666 --port=4500 --port=500 --port=3306 edge
似乎 Kubernetes 试图启动容器并 运行ning，但没有任何成功...
$kubectl get po
NAME READY REASON RESTARTS AGE
edge-sz7wp 0/1 Running 10 15m
$kubectl describe pod edge 命令给了我很多这样的东西：
Thu, 09 Nov 2017 17:13:05 +0000 Thu, 09 Nov 2017 17:13:05 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id abcc2ff25a624a998871e02bcb62d42d6f39e9db0a39f601efa4d357dd8334aa
Thu, 09 Nov 2017 17:13:15 +0000 Thu, 09 Nov 2017 17:13:15 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id 878778836bd3cc25bdf1e3b9cc2f2f6fa22b75b938a481172f08a6ec50571582
Thu, 09 Nov 2017 17:13:15 +0000 Thu, 09 Nov 2017 17:13:15 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id 878778836bd3cc25bdf1e3b9cc2f2f6fa22b75b938a481172f08a6ec50571582
Thu, 09 Nov 2017 17:13:25 +0000 Thu, 09 Nov 2017 17:13:25 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id aa51e94536216b905ff9ba07951fedbc0007476b55dfdb2e5106418fb6aee05c
Thu, 09 Nov 2017 17:13:25 +0000 Thu, 09 Nov 2017 17:13:25 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id aa51e94536216b905ff9ba07951fedbc0007476b55dfdb2e5106418fb6aee05c
Thu, 09 Nov 2017 17:13:35 +0000 Thu, 09 Nov 2017 17:13:35 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id f4661e5ea33471cd1ba30816b40c8ba2d204fa22509b973da4af6eedb64c592e
Thu, 09 Nov 2017 17:13:35 +0000 Thu, 09 Nov 2017 17:13:35 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id f4661e5ea33471cd1ba30816b40c8ba2d204fa22509b973da4af6eedb64c592e
Thu, 09 Nov 2017 17:13:45 +0000 Thu, 09 Nov 2017 17:13:45 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id 75f83dcb9b4f8af5134d6fd2edcd9342ecf56111e132a45f4e9787e83466e28b
Thu, 09 Nov 2017 17:13:45 +0000 Thu, 09 Nov 2017 17:13:45 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id 75f83dcb9b4f8af5134d6fd2edcd9342ecf56111e132a45f4e9787e83466e28b
Thu, 09 Nov 2017 17:13:55 +0000 Thu, 09 Nov 2017 17:13:55 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id c9d0535b3962ec9da29c068dbb0a6b64426a5ac3e52f72e79bcbaf03c9f3d403
Thu, 09 Nov 2017 17:13:55 +0000 Thu, 09 Nov 2017 17:13:55 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id c9d0535b3962ec9da29c068dbb0a6b64426a5ac3e52f72e79bcbaf03c9f3d403
Thu, 09 Nov 2017 17:14:05 +0000 Thu, 09 Nov 2017 17:14:05 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id 579f4428e9804404bd746cceee88bb6c73066a33263202bb5f1eb15f6ff26d7b
Thu, 09 Nov 2017 17:14:05 +0000 Thu, 09 Nov 2017 17:14:05 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id 579f4428e9804404bd746cceee88bb6c73066a33263202bb5f1eb15f6ff26d7b
Thu, 09 Nov 2017 17:14:15 +0000 Thu, 09 Nov 2017 17:14:15 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id d36b2c9ddf0b1a05d86b43d2a92eb3c00ae92d00e155d5a1be1da8e2682f901b
Thu, 09 Nov 2017 17:14:15 +0000 Thu, 09 Nov 2017 17:14:15 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id d36b2c9ddf0b1a05d86b43d2a92eb3c00ae92d00e155d5a1be1da8e2682f901b
Thu, 09 Nov 2017 17:14:25 +0000 Thu, 09 Nov 2017 17:14:25 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id 2d7b24537414f5e6f2981bf5f01596b19ea1abdb0eb4b81508fc7f44e8c34609
Thu, 09 Nov 2017 17:14:25 +0000 Thu, 09 Nov 2017 17:14:25 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id 2d7b24537414f5e6f2981bf5f01596b19ea1abdb0eb4b81508fc7f44e8c34609
Thu, 09 Nov 2017 17:14:35 +0000 Thu, 09 Nov 2017 17:14:35 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id fdae44c599b77d44839e4897b750203c183001a6053c926432ef5a3c7f4deb38
Thu, 09 Nov 2017 17:14:35 +0000 Thu, 09 Nov 2017 17:14:35 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id fdae44c599b77d44839e4897b750203c183001a6053c926432ef5a3c7f4deb38
Thu, 09 Nov 2017 17:14:45 +0000 Thu, 09 Nov 2017 17:14:45 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id 544351dda838d698e3bc125840edb6ad71cd0165a970cce46825df03b826eb38
Thu, 09 Nov 2017 17:14:45 +0000 Thu, 09 Nov 2017 17:14:45 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id 544351dda838d698e3bc125840edb6ad71cd0165a970cce46825df03b826eb38
Thu, 09 Nov 2017 17:14:55 +0000 Thu, 09 Nov 2017 17:14:55 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} created Created with docker id 00fe4c286c1cc9b905c9c0927f82b39d45d41295a9dd0852131bba087bb19610
Thu, 09 Nov 2017 17:14:55 +0000 Thu, 09 Nov 2017 17:14:55 +0000 1 {kubelet 127.0.0.1} spec.containers{edge} started Started with docker id 00fe4c286c1cc9b905c9c0927f82b39d45d41295a9dd0852131bba087bb19610
如有任何帮助，我们将不胜感激！
虽然我不能在没有重新生成能力和缺少日志的情况下得出结论，但很容易注意到的区别之一是您在 docker 命令中提供的权限，例如 NET_ADMIN 或 NET_RAW 等在 Kubernetes 运行 命令中缺失。
Kubernetes 还提供了在 pod 声明的 securityContext 中将此类权限分配给具有 capabilities 的 pod 的能力。
我不确定您是否可以使用 Kubectl 执行此操作，但如果您使用 Pod 的 YAML 声明，规范大致如下：
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: myshell
image: "ubuntu:14.04"
command:
- /bin/sleep
- "300"
securityContext:
capabilities:
add:
- NET_ADMIN
如需更多参考，我建议快速浏览一下：
致所有找不到答案的可怜人， pod一直重启的原因是它执行的命令已经退出，代码为0（表示成功）。
在我的例子中，我是 运行 /bin/bash 作为入口点命令，在我的 pod 配置 .yaml 文件中指定：
apiVersion: v1
kind: Pod
metadata:
name: edge
spec:
containers:
- name: edge
image: "crossense/edge:production"
command:
- /bin/bash
securityContext:
capabilities:
add:
- NET_ADMIN
- SYS_MODULE
- NET_RAW
volumeMounts:
- name: ip-nonlocal-bind
mountPath: /host/proc/sys/net/ipv4
- name: dev
mountPath: /host/dev
- name: modules
mountPath: /host/lib/modules
....
The solution was simply adding a non exiting command to the entrypoint. This can be any process run on foreground or simply a /bin/sleep
为了示例和以后的学习，我最终的 pod 配置文件如下所示：
apiVersion: v1
kind: Pod
metadata:
name: edge
spec:
hostNetwork: true
containers:
- name: edge
image: "crossense/edge:production"
command: ["/bin/bash", "-c"]
args: ["service rsyslog restart; service proxysql start; service mongodb start; service pdns-recursor start; service supervisor start; service danted start; touch /var/run/squid.pid; chown proxy /var/run/squid.pid; service squid restart; service ipsec start; /sbin/iptables-restore < /etc/iptables/rules.v4; sleep infinity"]
securityContext:
privileged: true
capabilities:
add:
- NET_ADMIN
- SYS_MODULE
- NET_RAW
volumeMounts:
- mountPath: /dev/shm
name: dshm
- name: ip-nonlocal-bind
mountPath: /host/proc/sys/net/ipv4
- name: dev
mountPath: /dev
- name: modules
mountPath: /lib/modules
ports:
- containerPort: 80
- containerPort: 8080
- containerPort: 53
protocol: UDP
- containerPort: 5353
protocol: UDP
- containerPort: 5666
- containerPort: 4500
- containerPort: 500
- containerPort: 3306
volumes:
- name: dshm
emptyDir:
medium: Memory
- name: ip-nonlocal-bind
hostPath:
path: /proc/sys/net/ipv4
- name: dev
hostPath:
path: /dev
type: Directory
- name: modules
hostPath:
path: /lib/modules
type: Directory
如有任何问题，请随时对此主题发表评论，或通过 max.vlashchuk@gmail.com 询问我 :)