无法将 shell 放入 kubernetes 中的城堡容器中
Unable to get a shell into citadel container in kubernetes
我使用 https://istio.io/docs/setup/kubernetes/helm-install 中的说明在 minikube 中安装了 Istio（包括 citadel）运行。
$ helm template install/kubernetes/helm/istio --name istio --namespace istio-system > $HOME/istio.yaml
$ kubectl create namespace istio-system
$ kubectl apply -f $HOME/istio.yaml
当我尝试将 shell 放入 citadel 容器时，出现错误：
$ kubectl exec -it istio-citadel-6d7f9c545b-bkvnx -- /bin/bash
OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"/bin/bash\": stat /bin/bash: no such file or directory": unknown
command terminated with exit code 126
不过，我可以像 pilot 一样执行到其他容器中。
这些是我的 pods 和容器，如果有帮助的话。
shell-demo: nginx,
istio-citadel-6d7f9c545b-bkvnx: docker.io/istio/citadel:1.0.3,
istio-cleanup-secrets-rp4wv: quay.io/coreos/hyperkube:v1.7.6_coreos.0,
istio-egressgateway-866885bb49-6jz9q: docker.io/istio/proxyv2:1.0.3,
istio-galley-6d74549bb9-7nhcl: docker.io/istio/galley:1.0.3,
istio-ingressgateway-6c6ffb7dc8-bvp6b: docker.io/istio/proxyv2:1.0.3,
istio-pilot-685fc95d96-fphc9: docker.io/istio/pilot:1.0.3, docker.io/istio/proxyv2:1.0.3,
istio-policy-688f99c9c4-bpl9w: docker.io/istio/mixer:1.0.3, docker.io/istio/proxyv2:1.0.3,
istio-security-post-install-s6dft: quay.io/coreos/hyperkube:v1.7.6_coreos.0,
istio-sidecar-injector-74855c54b9-6v5xg:docker.io/istio/sidecar_injector:1.0.3,
istio-telemetry-69b794ff59-f7dv4: docker.io/istio/mixer:1.0.3, docker.io/istio/proxyv2:1.0.3,
prometheus-f556886b8-lhdt8: docker.io/prom/prometheus:v2.3.1,
coredns-c4cffd6dc-6xblf: k8s.gcr.io/coredns:1.2.2,
etcd-minikube: k8s.gcr.io/etcd-amd64:3.1.12,
kube-addon-manager-minikube: k8s.gcr.io/kube-addon-manager:v8.6,
kube-apiserver-minikube: k8s.gcr.io/kube-apiserver-amd64:v1.10.0,
kube-controller-manager-minikube: k8s.gcr.io/kube-controller-manager-amd64:v1.10.0,
kube-dns-86f4d74b45-bjk54: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.8, k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.8, k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.8,
kube-proxy-mqfb9: k8s.gcr.io/kube-proxy-amd64:v1.10.0,
kube-scheduler-minikube: k8s.gcr.io/kube-scheduler-amd64:v1.10.0,
kubernetes-dashboard-6f4cfc5d87-zwk2c: k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.0,
storage-provisioner: gcr.io/k8s-minikube/storage-provisioner:v1.8.1,
当我执行 minikube ssh 然后尝试执行到 citadel 容器时，我收到类似的错误：
$ docker ps | grep citadel
f173453f843c istio/citadel "/usr/local/bin/isti…" 3 hours ago Up 3 hours k8s_citadel_istio-citadel-6d7f9c545b-bkvnx_istio-system_3d7b4f08-e120-11e8-bc40-ee7dbbb8f91b_0
7e96617d81ff k8s.gcr.io/pause-amd64:3.1 "/pause" 3 hours ago Up 3 hours k8s_POD_istio-citadel-6d7f9c545b-bkvnx_istio-system_3d7b4f08-e120-11e8-bc40-ee7dbbb8f91b_0
$ docker exec -it f173453f843c sh
OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"sh\": executable file not found in $PATH": unknown
$ docker exec -it f173453f843c /bin/sh
OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"/bin/sh\": stat /bin/sh: no such file or directory": unknown
$ docker exec -it f173453f843c ls
OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"ls\": executable file not found in $PATH": unknown
我可以看到 Citadel 容器日志正常。日志位于 https://pastebin.com/xTy9vSz2
你知道为什么我们不能执行到 citadel 容器吗？
感谢阅读。
您不能 shell 因为容器中 sh 和 bash 都不可用。很多时候，为了提高效率和拥有最小的容器镜像，这些都被删除了。
如果您想 shell 放入容器中，我建议您构建自己的映像，将 bash 或 sh 包含在其中。
你可以在这里看到 Dockerfile 构建了一个只有静态二进制文件的图像。为此，您想更改基本图像。例如：
FROM alpine
而不是：
FROM scratch
希望对您有所帮助。
Do you know why we can't exec into citadel container?
从下一个 Kubernetes 版本（1.16+，2019 年第 3 季度）开始，您可以。
参见 kubernetes/kubernetes PR 59416 (PR="pull request"): "Add Ephemeral Containers to the Kubernetes core API" (commit 7e6b70f)。
PR 79614 用于文档。
解决issue 27140：“支持distroless容器故障排除”。
（pnnl-miscscripts/miscscripts/bin 是一种解决方法）
An EphemeralContainer is a temporary container that may be added to an existing pod for
user-initiated activities such as debugging. Ephemeral containers have no resource or scheduling guarantees, and they will not be restarted when they exit or when a pod removed or restarted.
我使用 https://istio.io/docs/setup/kubernetes/helm-install 中的说明在 minikube 中安装了 Istio（包括 citadel）运行。
$ helm template install/kubernetes/helm/istio --name istio --namespace istio-system > $HOME/istio.yaml
$ kubectl create namespace istio-system
$ kubectl apply -f $HOME/istio.yaml
当我尝试将 shell 放入 citadel 容器时，出现错误：
$ kubectl exec -it istio-citadel-6d7f9c545b-bkvnx -- /bin/bash
OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"/bin/bash\": stat /bin/bash: no such file or directory": unknown
command terminated with exit code 126
不过，我可以像 pilot 一样执行到其他容器中。
这些是我的 pods 和容器，如果有帮助的话。
shell-demo: nginx,
istio-citadel-6d7f9c545b-bkvnx: docker.io/istio/citadel:1.0.3,
istio-cleanup-secrets-rp4wv: quay.io/coreos/hyperkube:v1.7.6_coreos.0,
istio-egressgateway-866885bb49-6jz9q: docker.io/istio/proxyv2:1.0.3,
istio-galley-6d74549bb9-7nhcl: docker.io/istio/galley:1.0.3,
istio-ingressgateway-6c6ffb7dc8-bvp6b: docker.io/istio/proxyv2:1.0.3,
istio-pilot-685fc95d96-fphc9: docker.io/istio/pilot:1.0.3, docker.io/istio/proxyv2:1.0.3,
istio-policy-688f99c9c4-bpl9w: docker.io/istio/mixer:1.0.3, docker.io/istio/proxyv2:1.0.3,
istio-security-post-install-s6dft: quay.io/coreos/hyperkube:v1.7.6_coreos.0,
istio-sidecar-injector-74855c54b9-6v5xg:docker.io/istio/sidecar_injector:1.0.3,
istio-telemetry-69b794ff59-f7dv4: docker.io/istio/mixer:1.0.3, docker.io/istio/proxyv2:1.0.3,
prometheus-f556886b8-lhdt8: docker.io/prom/prometheus:v2.3.1,
coredns-c4cffd6dc-6xblf: k8s.gcr.io/coredns:1.2.2,
etcd-minikube: k8s.gcr.io/etcd-amd64:3.1.12,
kube-addon-manager-minikube: k8s.gcr.io/kube-addon-manager:v8.6,
kube-apiserver-minikube: k8s.gcr.io/kube-apiserver-amd64:v1.10.0,
kube-controller-manager-minikube: k8s.gcr.io/kube-controller-manager-amd64:v1.10.0,
kube-dns-86f4d74b45-bjk54: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.8, k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.8, k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.8,
kube-proxy-mqfb9: k8s.gcr.io/kube-proxy-amd64:v1.10.0,
kube-scheduler-minikube: k8s.gcr.io/kube-scheduler-amd64:v1.10.0,
kubernetes-dashboard-6f4cfc5d87-zwk2c: k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.0,
storage-provisioner: gcr.io/k8s-minikube/storage-provisioner:v1.8.1,
当我执行 minikube ssh 然后尝试执行到 citadel 容器时，我收到类似的错误：
$ docker ps | grep citadel
f173453f843c istio/citadel "/usr/local/bin/isti…" 3 hours ago Up 3 hours k8s_citadel_istio-citadel-6d7f9c545b-bkvnx_istio-system_3d7b4f08-e120-11e8-bc40-ee7dbbb8f91b_0
7e96617d81ff k8s.gcr.io/pause-amd64:3.1 "/pause" 3 hours ago Up 3 hours k8s_POD_istio-citadel-6d7f9c545b-bkvnx_istio-system_3d7b4f08-e120-11e8-bc40-ee7dbbb8f91b_0
$ docker exec -it f173453f843c sh
OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"sh\": executable file not found in $PATH": unknown
$ docker exec -it f173453f843c /bin/sh
OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"/bin/sh\": stat /bin/sh: no such file or directory": unknown
$ docker exec -it f173453f843c ls
OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"ls\": executable file not found in $PATH": unknown
我可以看到 Citadel 容器日志正常。日志位于 https://pastebin.com/xTy9vSz2
你知道为什么我们不能执行到 citadel 容器吗？
感谢阅读。
您不能 shell 因为容器中 sh 和 bash 都不可用。很多时候，为了提高效率和拥有最小的容器镜像，这些都被删除了。
如果您想 shell 放入容器中，我建议您构建自己的映像，将 bash 或 sh 包含在其中。
你可以在这里看到 Dockerfile 构建了一个只有静态二进制文件的图像。为此，您想更改基本图像。例如：
FROM alpine
而不是：
FROM scratch
希望对您有所帮助。
Do you know why we can't exec into citadel container?
从下一个 Kubernetes 版本（1.16+，2019 年第 3 季度）开始，您可以。
参见 kubernetes/kubernetes PR 59416 (PR="pull request"): "Add Ephemeral Containers to the Kubernetes core API" (commit 7e6b70f)。
PR 79614 用于文档。
解决issue 27140：“支持distroless容器故障排除”。
（pnnl-miscscripts/miscscripts/bin 是一种解决方法）
An EphemeralContainer is a temporary container that may be added to an existing pod for user-initiated activities such as debugging. Ephemeral containers have no resource or scheduling guarantees, and they will not be restarted when they exit or when a pod removed or restarted.