我无法访问调度到另一个节点的 pod。但是我可以访问调度到当前节点的 pod

Question

我无法访问调度到另一个节点的 pod。但是我可以访问调度到当前节点的pod，反之亦然，当我在另一个节点上时，我只能访问调度到当前节点的pod，而不能访问调度到另一个节点的pod。并且当前节点上的路由规则与其他节点不同（事实上，我集群中的所有三个节点都有不同的路由规则）。下面列出了一些信息：

在主节点 172.16.5.150 上：

[root@localhost test-deploy]# kubectl get node
NAME           STATUS   ROLES    AGE   VERSION
172.16.5.150   Ready    <none>   9h    v1.16.2
172.16.5.151   Ready    <none>   9h    v1.16.2
172.16.5.152   Ready    <none>   9h    v1.16.2
[root@localhost test-deploy]# kubectl get pods --all-namespaces
NAMESPACE     NAME                                       READY   STATUS    RESTARTS   AGE
default       nginx-controller-5qvwn                     1/1     Running   0          46m
default       nginx-controller-kgjwm                     1/1     Running   0          46m
kube-system   calico-kube-controllers-6dbf77c57f-kcqtt   1/1     Running   0          33m
kube-system   calico-node-5zdt7                          1/1     Running   0          33m
kube-system   calico-node-8vqhv                          1/1     Running   0          33m
kube-system   calico-node-w9tq8                          1/1     Running   0          33m
kube-system   coredns-7b6b59774c-lzfh7                   1/1     Running   0          9h
[root@localhost test-deploy]#
[root@localhost test-deploy]# kcp -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP                NODE           NOMINATED NODE   READINESS GATES
nginx-controller-5qvwn   1/1     Running   0          23m   192.168.102.135   172.16.5.151   <none>           <none>
nginx-controller-kgjwm   1/1     Running   0          23m   192.168.102.134   172.16.5.150   <none>           <none>
[root@localhost test-deploy]# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         gateway         0.0.0.0         UG    100    0        0 ens32
172.0.0.0       0.0.0.0         255.0.0.0       U     100    0        0 ens32
192.168.102.128 0.0.0.0         255.255.255.192 U     0      0        0 *
192.168.102.129 0.0.0.0         255.255.255.255 UH    0      0        0 calia42aeb87aa8
192.168.102.134 0.0.0.0         255.255.255.255 UH    0      0        0 caliefbc513267b
[root@localhost test-deploy]# kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
kubernetes   ClusterIP   10.10.0.1       <none>        443/TCP    9h
nginx-svc    ClusterIP   10.10.189.192   <none>        8088/TCP   23m
[root@localhost test-deploy]# curl 192.168.102.135
curl: (7) Failed to connect to 192.168.102.135: 无效的参数
[root@localhost test-deploy]# curl 192.168.102.134
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@localhost test-deploy]# curl 10.10.189.192:8088
curl: (7) Failed connect to 10.10.189.192:8088; 没有到主机的路由
[root@localhost test-deploy]# curl 10.10.189.192:8088
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@localhost test-deploy]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:4b:76:b7 brd ff:ff:ff:ff:ff:ff
    inet 172.16.5.150/8 brd 172.255.255.255 scope global noprefixroute ens32
       valid_lft forever preferred_lft forever
    inet6 fe80::92f8:9957:1651:f41/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 12:00:37:16:be:95 brd ff:ff:ff:ff:ff:ff
4: kube-ipvs0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default 
    link/ether b2:9f:49:ff:31:3f brd ff:ff:ff:ff:ff:ff
    inet 10.10.0.1/32 brd 10.10.0.1 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.10.0.200/32 brd 10.10.0.200 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.10.189.192/32 brd 10.10.189.192 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
5: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1440 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
    inet 192.168.102.128/32 brd 192.168.102.128 scope global tunl0
       valid_lft forever preferred_lft forever
6: calia42aeb87aa8@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1440 qdisc noqueue state UP group default 
    link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::ecee:eeff:feee:eeee/64 scope link 
       valid_lft forever preferred_lft forever
7: caliefbc513267b@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1440 qdisc noqueue state UP group default 
    link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 1
    inet6 fe80::ecee:eeff:feee:eeee/64 scope link 
       valid_lft forever preferred_lft forever

在另一个节点 172.16.5.150 上：

[root@localhost ~]# curl 10.10.189.192:8088
curl: (7) Failed connect to 10.10.189.192:8088; 没有到主机的路由
[root@localhost ~]# curl 10.10.189.192:8088
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@localhost ~]# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         gateway         0.0.0.0         UG    100    0        0 ens192
172.16.5.0      0.0.0.0         255.255.255.0   U     100    0        0 ens192
192.168.102.128 0.0.0.0         255.255.255.192 U     0      0        0 *
192.168.102.135 0.0.0.0         255.255.255.255 UH    0      0        0 cali44ab0f7df0f
[root@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:0c:29:38:a2:95 brd ff:ff:ff:ff:ff:ff
    inet 172.16.5.151/24 brd 172.16.5.255 scope global noprefixroute ens192
       valid_lft forever preferred_lft forever
    inet6 fe80::e24a:6e5c:3a44:a7ee/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 76:91:46:b1:06:a7 brd ff:ff:ff:ff:ff:ff
4: kube-ipvs0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default 
    link/ether 1a:0d:f4:cf:ab:69 brd ff:ff:ff:ff:ff:ff
    inet 10.10.0.1/32 brd 10.10.0.1 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.10.0.200/32 brd 10.10.0.200 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.10.189.192/32 brd 10.10.189.192 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
5: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1440 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
    inet 192.168.102.128/32 brd 192.168.102.128 scope global tunl0
       valid_lft forever preferred_lft forever
8: cali44ab0f7df0f@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1440 qdisc noqueue state UP group default 
    link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::ecee:eeff:feee:eeee/64 scope link 
       valid_lft forever preferred_lft forever

Answer 1

路由 table 没有 tunl0 接口的路由。您可以在 calico-node 部分下的 calico.yaml 文件中包含环境 IP_AUTODETECTION_METHOD。

Example:
 containers:
 - name: calico-node
   image: xxxxxxx
   env:
    - name: IP_AUTODETECTION_METHOD
      value: interface=ens192

我无法访问调度到另一个节点的 pod。但是我可以访问调度到当前节点的 pod

I can't access the pod which scheduled to the another node. But i can access the pod which scheduled to the current node

calico

kubernetes

project-calico