如何从另一个 docker 容器访问类控制平面端口?

How to access kind control plane port from another docker container?

我正在使用 kind create cluster --name kind 创建一个 kind 集群,我想从另一个 docker 容器访问它,但是当我尝试从容器应用 Kubernetes 文件时(kubectl apply -f deployment.yml ) 我得到了这个错误:

The connection to the server 127.0.0.1:6445 was refused - did you specify the right host or port?

确实,当我尝试从容器中 curl kind control-plane 时,它​​无法访问。

> docker run --entrypoint curl curlimages/curl:latest 127.0.0.1:6445
curl: (7) Failed to connect to 127.0.0.1 port 6445 after 0 ms: Connection refused

无论哪种控制平面都会发布到正确的端口,但只发布到本地主机。

> docker ps --format "table {{.Image}}\t{{.Ports}}"
IMAGE                  PORTS
kindest/node:v1.23.4   127.0.0.1:6445->6443/tcp

目前我找到的唯一解决方案是设置主机网络模式。

> docker run --network host --entrypoint curl curlimages/curl:latest 127.0.0.1:6445
Client sent an HTTP request to an HTTPS server.

这个解决方案看起来不是最安全的。有没有其他方法,比如将 kind 网络连接到我的容器或我错过的类似方法?

我不知道你为什么要这样做。但没问题,我认为这可以帮助你:

首先,让我们拉取您的 docker 图像:

❯ docker pull curlimages/curl

在我的 kind 集群中,我有 3 个控制平面节点和 3 个工作节点。以下是我的 kind 集群的 pod:

❯ docker ps
CONTAINER ID   IMAGE                                COMMAND                  CREATED          STATUS          PORTS                       NAMES
39dbbb8ca320   kindest/node:v1.23.5                 "/usr/local/bin/entr…"   7 days ago       Up 7 days       127.0.0.1:35327->6443/tcp   so-cluster-1-control-plane
62b5538275e9   kindest/haproxy:v20220207-ca68f7d4   "haproxy -sf 7 -W -d…"   7 days ago       Up 7 days       127.0.0.1:35625->6443/tcp   so-cluster-1-external-load-balancer
9f189a1b6c52   kindest/node:v1.23.5                 "/usr/local/bin/entr…"   7 days ago       Up 7 days       127.0.0.1:40845->6443/tcp   so-cluster-1-control-plane3
4c53f745a6ce   kindest/node:v1.23.5                 "/usr/local/bin/entr…"   7 days ago       Up 7 days       127.0.0.1:36153->6443/tcp   so-cluster-1-control-plane2
97e5613d2080   kindest/node:v1.23.5                 "/usr/local/bin/entr…"   7 days ago       Up 7 days       0.0.0.0:30081->30080/tcp    so-cluster-1-worker2
0ca64a907707   kindest/node:v1.23.5                 "/usr/local/bin/entr…"   7 days ago       Up 7 days       0.0.0.0:30080->30080/tcp    so-cluster-1-worker
9c5d26caee86   kindest/node:v1.23.5                 "/usr/local/bin/entr…"   7 days ago       Up 7 days       0.0.0.0:30082->30080/tcp    so-cluster-1-worker3

这里我们感兴趣的容器是 haproxy 容器 (kindest/haproxy:v20220207-ca68f7d4),它的作用是对进入节点的流量进行负载均衡(在我们的示例中,尤其是控制平面节点。)我们可以看到我们主机的35625端口映射到haproxy容器的6443端口。 (127.0.0.1:35625->6443/tcp)

所以,我们的集群端点是 https://127.0.0.1:35625,我们可以在我们的 kubeconfig 文件 (~/.kube/config) 中确认这一点:

❯ cat .kube/config
apiVersion: v1
kind: Config
preferences: {}
users:
  - name: kind-so-cluster-1
    user:
        client-certificate-data: <base64data>
        client-key-data: <base64data>
clusters:
  - cluster:
        certificate-authority-data: <certificate-authority-dataBase64data>
        server: https://127.0.0.1:35625
    name: kind-so-cluster-1
contexts:
  - context:
        cluster: kind-so-cluster-1
        user: kind-so-cluster-1
        namespace: so-tests
    name: kind-so-cluster-1
current-context: kind-so-cluster-1

让我们运行后台的curl容器:

❯ docker run -d --network host curlimages/curl sleep 3600
ba183fe2bb8d715ed1e503a9fe8096dba377f7482635eb12ce1322776b7e2366

正如预期的那样,我们无法通过 HTTP 请求侦听 HTTPS 端口的端点:

❯  docker exec -it ba curl 127.0.0.1:35625
Client sent an HTTP request to an HTTPS server.

我们可以尝试使用我们 kubeconfig 中“certificate-authority-data”字段中的证书来检查它是否改变了什么(它应该): 让我们创建一个名为 my-ca.crt 的文件,其中包含证书的字符串数据:

base64 -d <<<  <certificate-authority-dataBase64dataFromKubeConfig> > my-ca.crt 

因为 curl docker 图像的工作目录是“/”让我们将我们的证书复制到容器中的这个位置并验证它确实存在:

docker cp my-ca.crt ba183fe:/

❯ docker exec -it ba sh
/ $ ls my-ca.crt
my-ca.crt

让我们再次尝试我们的 curl 请求,但使用证书:

❯ docker exec -it ba curl --cacert my-ca.crt https://127.0.0.1:35625
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "forbidden: User \"system:anonymous\" cannot get path \"/\"",
  "reason": "Forbidden",
  "details": {},
  "code": 403
}

您可以通过在您的 curl 请求中添加“--insecure”标志来获得相同的结果:

❯  docker exec -it ba curl https://127.0.0.1:35625 --insecure
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "forbidden: User \"system:anonymous\" cannot get path \"/\"",
  "reason": "Forbidden",
  "details": {},
  "code": 403
}

但是,我们无法使用匿名用户访问我们的集群!因此,让我们从 kubernetes 获取令牌(cf https://kubernetes.io/docs/tasks/administer-cluster/access-cluster-api/):

# Create a secret to hold a token for the default service account
kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: default-token
  annotations:
    kubernetes.io/service-account.name: default
type: kubernetes.io/service-account-token
EOF

一旦令牌控制器用令牌填充了秘密:

# Get the token value
❯ kubectl get secret default-token -o jsonpath='{.data.token}' | base64 --decode
eyJhbGciOiJSUzI1NiIsImtpZCI6InFSTThZZ05lWHFXMWExQlVSb1hTcHNxQ3F6Z2Z2aWpUaUYwd2F2TGdVZ0EifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJzby10ZXN0cyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIzYzY0OTg1OS0xNzkyLTQzYTQtOGJjOC0zMDEzZDgxNjRmY2IiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6c28tdGVzdHM6ZGVmYXVsdCJ9.VLfjuym0fohYTT_uoLPwM0A6u7dUt2ciWZF2K9LM_YvQ0UZT4VgkM8UBVOQpWjTmf9s2B5ZxaOkPu4cz_B4xyDLiiCgqiHCbUbjxE9mphtXGKQwAeKLvBlhbjYnHb9fCTRW19mL7VhqRgfz5qC_Tae7ysD3uf91FvqjjxsCyzqSKlsq0T7zXnzQ_YQYoUplGa79-LS_xDwG-2YFXe0RfS9hkpCILpGDqhLXci_gwP9DW0a6FM-L1R732OdGnb9eCPI6ReuTXQz7naQ4RQxZSIiNd_S7Vt0AYEg-HGvSkWDl0_DYIyHShMeFHu1CtfTZS5xExoY4-_LJD8mi

现在让我们直接使用令牌执行 curl 命令!

❯ docker exec -it ba curl -X GET https://127.0.0.1:35625/api --header "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6InFSTThZZ05lWHFXMWExQlVSb1hTcHNxQ3F6Z2Z2aWpUaUYwd2F2TGdVZ0EifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJzby10ZXN0cyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIzYzY0OTg1OS0xNzkyLTQzYTQtOGJjOC0zMDEzZDgxNjRmY2IiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6c28tdGVzdHM6ZGVmYXVsdCJ9.VLfjuym0fohYTT_uoLPwM0A6u7dUt2ciWZF2K9LM_YvQ0UZT4VgkM8UBVOQpWjTmf9s2B5ZxaOkPu4cz_B4xyDLiiCgqiHCbUbjxE9mphtXGKQwAeKLvBlhbjYnHb9fCTRW19mL7VhqRgfz5qC_Tae7ysD3uf91FvqjjxsCyzqSKlsq0T7zXnzQ_YQYoUplGa79-LS_xDwG-2YFXe0RfS9hkpCILpGDqhLXci_gwP9DW0a6FM-L1R732OdGnb9eCPI6ReuTXQz7naQ4RQxZSIiNd_S7Vt0AYEg-HGvSkWDl0_DYIyHShMeFHu1CtfTZS5xExoY4-_LJD8mi" --insecure
{
  "kind": "APIVersions",
  "versions": [
    "v1"
  ],
  "serverAddressByClientCIDRs": [
    {
      "clientCIDR": "0.0.0.0/0",
      "serverAddress": "172.18.0.5:6443"
    }
  ]
}

有效! 我仍然不知道你为什么要这样做,但我希望这对你有所帮助。

因为这不是你想要的,因为我在这里使用主机网络,你可以使用这个: 建议@SergioSantiago 感谢你的评论!

bguess