Kubernetes DNS 不解析外部地址
Kubernetes DNS not resolving external addresses
我的 kubernetes 集群和 DNS 有问题。
我们最近更新到 RHEL 7.5,其中一台机器是 Master。一旦它重新联机,大多数都可以正常工作,但我只是注意到来自集群的外部连接无法解析。内部沟通效果很好。
这是 busybox nslookup 结果:
kubectl exec -ti busybox -- nslookup kubernetes.default
Server: 10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
nslookup: can't resolve 'kubernetes.default'
kubedns 状态
Name: kube-dns
Namespace: kube-system
Labels: k8s-app=kube-dns
kubernetes.io/cluster-service=true
kubernetes.io/name=KubeDNS
Annotations: <none>
Selector: k8s-app=kube-dns
Type: ClusterIP
IP: 10.96.0.10
Port: dns 53/UDP
TargetPort: 53/UDP
Endpoints: 10.32.0.18:53
Port: dns-tcp 53/TCP
TargetPort: 53/TCP
Endpoints: 10.32.0.18:53
Session Affinity: None
Events: <none>
所有 pods 都说他们起来了
NAME READY STATUS RESTARTS AGE
kube-dns-86f4d74b45-9m292 3/3 Running 26 44d
来自 kubedns 的最新日志 pods
I0523 15:59:14.291623 1 logs.go:41] skydns: ready for queries on cluster.local. for tcp://0.0.0.0:10053 [rcache 0]
I0523 15:59:14.291638 1 logs.go:41] skydns: ready for queries on cluster.local. for udp://0.0.0.0:10053 [rcache 0]
I0523 15:59:14.791440 1 dns.go:170] Initialized services and endpoints from apiserver
I0523 15:59:14.791560 1 server.go:135] Setting up Healthz Handler (/readiness)
I0523 15:59:14.791579 1 server.go:140] Setting up cache handler (/cache)
I0523 15:59:14.791588 1 server.go:126] Status HTTP port 8081
I0523 16:48:05.175159 1 dns.go:555] Could not find endpoints for service "kube-prometheus-exporter-kube-controller-manager" in namespace "kube-system". DNS records will be created once endpoints show up.
I0523 16:48:05.236418 1 dns.go:555] Could not find endpoints for service "kube-prometheus-exporter-kube-dns" in namespace "kube-system". DNS records will be created once endpoints show up.
I0523 16:48:05.252139 1 dns.go:555] Could not find endpoints for service "kube-prometheus-exporter-kube-etcd" in namespace "kube-system". DNS records will be created once endpoints show up.
I0523 16:48:05.264292 1 dns.go:555] Could not find endpoints for service "kube-prometheus-exporter-kube-scheduler" in namespace "kube-system". DNS records will be created once endpoints show up.
I0524 13:04:38.008038 1 logs.go:41] skydns: failure to forward request "read udp 10.32.0.18:48431->10.101.24.100:53: i/o timeout"
I0524 13:04:41.973900 1 logs.go:41] skydns: failure to forward request "read udp 10.32.0.18:45738->10.101.24.100:53: i/o timeout"
I0524 17:26:54.056913 1 logs.go:41] skydns: failure to forward request "read udp 10.32.0.18:52242->10.101.24.100:53: i/o timeout"
I0524 18:28:16.762002 1 logs.go:41] skydns: failure to forward request "read udp 10.32.0.18:43051->10.101.24.100:53: i/o timeout"
I0524 18:28:20.762912 1 logs.go:41] skydns: failure to forward request "read udp 10.32.0.18:57624->10.101.24.100:53: i/o timeout"
我假设这与重启有关。
Kubernetes:版本 1.10.3
DNS:kube-dns
网络:Weavenet
主机OS:RHEL 7.5
任何帮助都会很棒。
谢谢
不确定升级的详细信息,但请检查升级 OS 时您的 kube-dns 版本是否更改。 kube-dns 1.14.9 版本在解析外部名称服务时存在问题。您需要将 kube-dns 回滚到以前的版本,或者升级到 1.14.10。
我的 kubernetes 集群和 DNS 有问题。
我们最近更新到 RHEL 7.5,其中一台机器是 Master。一旦它重新联机,大多数都可以正常工作,但我只是注意到来自集群的外部连接无法解析。内部沟通效果很好。
这是 busybox nslookup 结果:
kubectl exec -ti busybox -- nslookup kubernetes.default
Server: 10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
nslookup: can't resolve 'kubernetes.default'
kubedns 状态
Name: kube-dns
Namespace: kube-system
Labels: k8s-app=kube-dns
kubernetes.io/cluster-service=true
kubernetes.io/name=KubeDNS
Annotations: <none>
Selector: k8s-app=kube-dns
Type: ClusterIP
IP: 10.96.0.10
Port: dns 53/UDP
TargetPort: 53/UDP
Endpoints: 10.32.0.18:53
Port: dns-tcp 53/TCP
TargetPort: 53/TCP
Endpoints: 10.32.0.18:53
Session Affinity: None
Events: <none>
所有 pods 都说他们起来了
NAME READY STATUS RESTARTS AGE
kube-dns-86f4d74b45-9m292 3/3 Running 26 44d
来自 kubedns 的最新日志 pods
I0523 15:59:14.291623 1 logs.go:41] skydns: ready for queries on cluster.local. for tcp://0.0.0.0:10053 [rcache 0]
I0523 15:59:14.291638 1 logs.go:41] skydns: ready for queries on cluster.local. for udp://0.0.0.0:10053 [rcache 0]
I0523 15:59:14.791440 1 dns.go:170] Initialized services and endpoints from apiserver
I0523 15:59:14.791560 1 server.go:135] Setting up Healthz Handler (/readiness)
I0523 15:59:14.791579 1 server.go:140] Setting up cache handler (/cache)
I0523 15:59:14.791588 1 server.go:126] Status HTTP port 8081
I0523 16:48:05.175159 1 dns.go:555] Could not find endpoints for service "kube-prometheus-exporter-kube-controller-manager" in namespace "kube-system". DNS records will be created once endpoints show up.
I0523 16:48:05.236418 1 dns.go:555] Could not find endpoints for service "kube-prometheus-exporter-kube-dns" in namespace "kube-system". DNS records will be created once endpoints show up.
I0523 16:48:05.252139 1 dns.go:555] Could not find endpoints for service "kube-prometheus-exporter-kube-etcd" in namespace "kube-system". DNS records will be created once endpoints show up.
I0523 16:48:05.264292 1 dns.go:555] Could not find endpoints for service "kube-prometheus-exporter-kube-scheduler" in namespace "kube-system". DNS records will be created once endpoints show up.
I0524 13:04:38.008038 1 logs.go:41] skydns: failure to forward request "read udp 10.32.0.18:48431->10.101.24.100:53: i/o timeout"
I0524 13:04:41.973900 1 logs.go:41] skydns: failure to forward request "read udp 10.32.0.18:45738->10.101.24.100:53: i/o timeout"
I0524 17:26:54.056913 1 logs.go:41] skydns: failure to forward request "read udp 10.32.0.18:52242->10.101.24.100:53: i/o timeout"
I0524 18:28:16.762002 1 logs.go:41] skydns: failure to forward request "read udp 10.32.0.18:43051->10.101.24.100:53: i/o timeout"
I0524 18:28:20.762912 1 logs.go:41] skydns: failure to forward request "read udp 10.32.0.18:57624->10.101.24.100:53: i/o timeout"
我假设这与重启有关。
Kubernetes:版本 1.10.3
DNS:kube-dns
网络:Weavenet
主机OS:RHEL 7.5
任何帮助都会很棒。
谢谢
不确定升级的详细信息,但请检查升级 OS 时您的 kube-dns 版本是否更改。 kube-dns 1.14.9 版本在解析外部名称服务时存在问题。您需要将 kube-dns 回滚到以前的版本,或者升级到 1.14.10。