Zookeeper：主机名解析失败

Question

我是运行 Zookeeper，在 OpenShift/Kubernetes 环境中。我将 zookeeper 设置为 StatefulSet 以便可靠地保留配置数据。

我在 zoo.cfg 中通过主机名配置了三个服务器，但在启动时，主机名解析失败。我验证了主机名确实可以在我的集群中使用 nslookup 解析。

zoo.cfg:

clientPort=2181
dataDir=/var/lib/zookeeper/data
dataLogDir=/var/lib/zookeeper/log
tickTime=2000
initLimit=10
syncLimit=2000
maxClientCnxns=60
minSessionTimeout= 4000
maxSessionTimeout= 40000
autopurge.snapRetainCount=3
autopurge.purgeInteval=0
server.1=zookeeper-0.zookeeper-headless:2888:3888
server.2=zookeeper-1.zookeeper-headless:2888:3888
server.3=zookeeper-2.zookeeper-headless:2888:3888

我的 OpenShift/Kubernetes 配置的相关部分：

  # StatefulSet
  - apiVersion: apps/v1beta1
    kind: StatefulSet
    metadata:
      labels:
        app: zookeeper
      name: zookeeper
    spec:
      serviceName: zookeeper-headless
      replicas: 3
      template:
        metadata:
          labels:
            app: zookeeper
        spec:
          containers:
            - image: 172.30.158.156:5000/os-cloud-platform/zookeeper:latest
              name: zookeeper
              ports:
                - containerPort: 2181
                  protocol: TCP
                  name: client
                - containerPort: 2888
                  protocol: TCP
                  name: server
                - containerPort: 3888
                  protocol: TCP
                  name: leader-election
          dnsPolicy: ClusterFirst
          schedulerName: default-scheduler

  # Service
  - apiVersion: v1
    kind: Service
    metadata:
      labels:
        app: zookeeper
      name: zookeeper
    spec:
      ports:
        - name: client
          port: 2181
          protocol: TCP
          targetPort: 2181
      selector:
        app: zookeeper
      sessionAffinity: None
      type: ClusterIP

  - apiVersion: v1
    kind: Service
    metadata:
      name: zookeeper-headless
      labels:
        app: zookeeper
    spec:
      ports:
        - port: 2888
          name: server
        - port: 3888
          name: leader-election
      clusterIP: None
      selector:
        app: zookeeper

OpenShift 日志显示 UnknownHostExceptions，但：

2017-10-06 10:59:18,289 [myid:] - WARN  [main:QuorumPeer$QuorumServer@155] - Failed to resolve address: zookeeper-2.zookeeper-headless
java.net.UnknownHostException: zookeeper-2.zookeeper-headless: No address associated with hostname
    at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
    at java.net.InetAddress.lookupAllHostAddr(InetAddress.java:928)
    at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
    at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
    at java.net.InetAddress.getAllByName(InetAddress.java:1192)
    at java.net.InetAddress.getAllByName(InetAddress.java:1126)
    at java.net.InetAddress.getByName(InetAddress.java:1076)
    at org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer.recreateSocketAddresses(QuorumPeer.java:148)
    at org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer.<init>(QuorumPeer.java:133)
    at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:228)
    at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:140)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:101)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
...

可能是什么原因？我验证了主机名（例如 zookeeper-2.zookeeper-headless）可从其他 pods 到 nslookup.

获得

Answer 1

我找到了解决此问题的可行方法。 ZooKeeper 在启动时读取集合中的服务器列表并查找其 "own" 条目。然后它使用此条目来确定要侦听的端口和接口。

server.1=zookeeper-0.zookeeper-headless:2888:3888
server.2=zookeeper-1.zookeeper-headless:2888:3888
server.3=zookeeper-2.zookeeper-headless:2888:3888

由于提供的主机名将在此机器上解析为 127.0.0.1，ZooKeeper 将侦听本地环回接口，因此不接受来自其他 ZooKeeper 服务器的连接。

server.1=0.0.0.0:2888:3888
server.2=zookeeper-1.zookeeper-headless:2888:3888
server.3=zookeeper-2.zookeeper-headless:2888:3888

为了在集群中实现自动化，我写了一个 bash 脚本来替换容器启动时的 "own" 条目。

编辑： 正如评论中所问，这是我的 ENTRYPOINT 脚本，它负责放置 myid 文件并为每个 [=14] 设置适当的主机名=]:

#!/bin/bash
# This script extracts the number out of the pod's hostname and sets it as zookeepers id.

# Exact paths may vary according to your setup
MYID_FILE="/var/lib/zookeeper/data/myid"
ZOOCFG_FILE="/conf/zoo.cfg"

# Create myid-file
# Extract only numbers from hostname
id=$(hostname | tr -d -c 0-9)
echo $id > "${MYID_FILE}"

# change own hostname to 0.0.0.0
# otherwise, the own hostname will resolve to 127.0.0.1
# 
fullHostname="$(hostname).zookeeper-headless"
sed -i -e "s/${fullHostname}/0.0.0.0/g" "${ZOOCFG_FILE}"

echo "Executing $@"
exec "$@"

Zookeeper：主机名解析失败

Zookeeper: Hostname resolution fails

kubernetes

openshift

apache-zookeeper

kube-dns