库伯内斯。无法从 master 连接到任何 pod

Question

我正在尝试在 Openstack + CoreOS 中设置 Kubernetes。

我有 master 10.240.63.84 和 2 个 minions .63 和 .83。我还创建了 3 个 redis pods:

redis-gopher-gziey        10.244.32.2     10.240.63.66/10.240.63.66
redis-managed-oh43e   10.244.32.3     10.240.63.66/10.240.63.66
redis-primary-fplln          10.244.54.2     10.240.63.83/10.240.63.83

大师的路由table看起来像：

10.240.63.0     *               255.255.255.0   U     0      0        0 eth0
10.240.63.1     *               255.255.255.255 UH    1024   0        0 eth0
10.244.0.0      *               255.255.0.0     U     0      0        0 flannel.1
10.244.50.0     *               255.255.255.0   U     0      0        0 docker0

ifconfig -a 的输出是：

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 10.244.50.1  netmask 255.255.255.0  broadcast 0.0.0.0
        inet6 fe80::542f:6fff:fe4a:adf3  prefixlen 64  scopeid 0x20<link>
        ether 56:84:7a:fe:97:99  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1  bytes 90 (90.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500  
        inet 10.240.63.84  netmask 255.255.255.0  broadcast 10.240.63.255
        inet6 fe80::f816:3eff:fe89:e9a0  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:89:e9:a0  txqueuelen 1000  (Ethernet)
        RX packets 430706  bytes 559764129 (533.8 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 238519  bytes 116083693 (110.7 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 10.244.50.0  netmask 255.255.0.0  broadcast 0.0.0.0
        inet6 fe80::601f:62ff:feed:1556  prefixlen 64  scopeid 0x20<link>
        ether 62:1f:62:ed:15:56  txqueuelen 0  (Ethernet)
        RX packets 20  bytes 1504 (1.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 79  bytes 7686 (7.5 KiB)
        TX errors 0  dropped 19 overruns 0  carrier 0  collisions 0

用于初始化的 Flanneld 配置为：

大师：

    - name: flanneld.service
      command: start
      drop-ins:
        - name: 50-network-config.conf
          content: |
            [Service]
            ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"10.244.0.0/16", "Backend": {"Type": "vxlan"}}'
            ExecStart=
            ExecStart=/usr/libexec/sdnotify-proxy /run/flannel/sd.sock \
                /usr/bin/docker run --net=host --privileged=true --rm \
                    --volume=/run/flannel:/run/flannel \
                    --env=NOTIFY_SOCKET=/run/flannel/sd.sock \
                    --env-file=/run/flannel/options.env \
                    --volume=${ETCD_SSL_DIR}:/etc/ssl/etcd:ro \
                    quay.io/coreos/flannel:${FLANNEL_VER} /opt/bin/flanneld --ip-masq=true --iface=eth0

小兵：

    - name: flanneld.service
      command: start
      drop-ins:
        - name: 50-network-config.conf
          content: |
            [Service]
            ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"10.244.0.0/16", "Backend": {"Type": "vxlan"}}'
            ExecStart=
            ExecStart=/usr/libexec/sdnotify-proxy /run/flannel/sd.sock \
                /usr/bin/docker run --net=host --privileged=true --rm \
                    --volume=/run/flannel:/run/flannel \
                    --env=NOTIFY_SOCKET=/run/flannel/sd.sock \
                    --env-file=/run/flannel/options.env \
                    --volume=${ETCD_SSL_DIR}:/etc/ssl/etcd:ro \
                    quay.io/coreos/flannel:${FLANNEL_VER} /opt/bin/flanneld -etcd-endpoints     http://10.240.63.84:4001 --ip-masq=true --iface=eth0

所以问题是我无法从 master ping 任何 pods，也无法连接到任何端口，错误是：

ncat -v -t 10.244.32.2 6379
Ncat: Version 6.40 ( http://nmap.org/ncat )
Ncat: No route to host.

Answer 1

这种事情很难远程调试。我要检查的东西：

1) 发件人：iptables -t raw -I OUTPUT -d 10.244.32.2 -j TRACE; dmesg -c > /dev/null; ncat -v -t 10.244.32.2 6379; dmesg;

这会让您深入了解内核正在做什么。

2) 在发件人上：tcpdump -i any host 10.244.32.2 & ncat -v -t 10.244.32.2 6379;`

这将提供更多见解。

3) 在接收器上：iptables -t raw -I OUTPUT -d 10.244.32.2 -j TRACE; dmesg -c > /dev/null; ncat -v -t 10.244.32.2 6379; dmesg;

这将告诉您数据包是否通过封装。

你需要通过整个连接来基本证明管道。

库伯内斯。无法从 master 连接到任何 pod

Kubernetes. Unable connect to any pod from master

openstack

coreos

kubernetes