使用 calico 在 kubernetes 中错过了路线

Question

我正在 centos8 上安装带有 calico 的 k8s，一切看起来都很好，但我无法在 pods.

之间相互 ping 通

我用的是k8s作为calico的DATASTORE，部署文件在calico.yaml

不知道为什么会漏掉一些路线，欢迎大家指点。

以下是集群的一些信息：

# kubectl get nodes
NAME                  STATUS   ROLES    AGE     VERSION
instance-4njec0xa-1   Ready    <none>   3h55m   v1.19.4
instance-4njec0xa-2   Ready    <none>   3h55m   v1.19.4
instance-4njec0xa-3   Ready    master   3h56m   v1.19.4

在主节点上

# ./calicoctl node status
Calico process is running.

IPv4 BGP status
+---------------+-------------------+-------+----------+-------------+
| PEER ADDRESS  |     PEER TYPE     | STATE |  SINCE   |    INFO     |
+---------------+-------------------+-------+----------+-------------+
| 192.168.0.194 | node-to-node mesh | up    | 04:10:41 | Established |
| 192.168.0.195 | node-to-node mesh | up    | 04:10:41 | Established |
+---------------+-------------------+-------+----------+-------------+

IPv6 BGP status
No IPv6 peers found.

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG    100    0        0 eth0
169.254.169.254 192.168.0.2     255.255.255.255 UGH   100    0        0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.17.139.128  0.0.0.0         255.255.255.255 UH    0      0        0 cali96f39d92828
172.17.139.128  0.0.0.0         255.255.255.192 U     0      0        0 *
172.17.139.129  0.0.0.0         255.255.255.255 UH    0      0        0 caliccf893b1917
172.17.139.130  0.0.0.0         255.255.255.255 UH    0      0        0 cali09dc1beebda
172.17.153.64   192.168.0.194   255.255.255.192 UG    0      0        0 eth0
172.17.181.64   192.168.0.195   255.255.255.192 UG    0      0        0 eth0
192.168.0.0     0.0.0.0         255.255.240.0   U     100    0        0 eth0

在节点 1 上

# ./calicoctl node status
Calico process is running.

IPv4 BGP status
+---------------+-------------------+-------+----------+-------------+
| PEER ADDRESS  |     PEER TYPE     | STATE |  SINCE   |    INFO     |
+---------------+-------------------+-------+----------+-------------+
| 192.168.0.195 | node-to-node mesh | up    | 04:10:42 | Established |
| 192.168.0.196 | node-to-node mesh | up    | 04:10:40 | Established |
+---------------+-------------------+-------+----------+-------------+

IPv6 BGP status
No IPv6 peers found.

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG    100    0        0 eth0
169.254.169.254 192.168.0.2     255.255.255.255 UGH   100    0        0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.17.139.128  192.168.0.196   255.255.255.192 UG    0      0        0 eth0
172.17.153.64   0.0.0.0         255.255.255.192 U     0      0        0 *
172.17.153.69   0.0.0.0         255.255.255.255 UH    0      0        0 cali2587d39bec8
172.17.181.64   192.168.0.195   255.255.255.192 UG    0      0        0 eth0
192.168.0.0     0.0.0.0         255.255.240.0   U     100    0        0 eth0

节点 2

./calicoctl node status
Calico process is running.

IPv4 BGP status
+---------------+-------------------+-------+----------+-------------+
| PEER ADDRESS  |     PEER TYPE     | STATE |  SINCE   |    INFO     |
+---------------+-------------------+-------+----------+-------------+
| 192.168.0.194 | node-to-node mesh | up    | 04:10:42 | Established |
| 192.168.0.196 | node-to-node mesh | up    | 04:10:40 | Established |
+---------------+-------------------+-------+----------+-------------+

IPv6 BGP status
No IPv6 peers found.

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG    100    0        0 eth0
169.254.169.254 192.168.0.2     255.255.255.255 UGH   100    0        0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.17.139.128  192.168.0.196   255.255.255.192 UG    0      0        0 eth0
172.17.153.64   192.168.0.194   255.255.255.192 UG    0      0        0 eth0
172.17.181.64   0.0.0.0         255.255.255.192 U     0      0        0 *
172.17.181.66   0.0.0.0         255.255.255.255 UH    0      0        0 cali12d4a061371
192.168.0.0     0.0.0.0         255.255.240.0   U     100    0        0 eth0

Answer 1

calico file默认关闭ipip和vxlan，导致网络不通。当我启用 ipip 并设置正确的 veth_mtu 时，一切正常。

使用 calico 在 kubernetes 中错过了路线

route missed in kubernetes with calico

networking

kubernetes

project-calico