Kubernetes 节点无法访问外部 Kafka 集群

Question

我有一个 Kubernetes 集群，其中包含多个 java 需要连接到远程 Kafka 集群的微服务。服务器位于 DigitalOcean 中，并且可以在其专用网络内完全访问。 Kafka集群没有使用Kubernetes，也不在集群内。

我使用 kubeadm 启动了整个集群，这是关于集群的信息：

# kubectl cluster-info
Kubernetes master is running at https://10.132.113.68:6443
KubeDNS is running at https://10.132.113.68:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

集群版本

# kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.2", GitCommit:"81753b10df112992bf51bbc2c2f85208aad78335", GitTreeState:"clean", BuildDate:"2018-04-27T09:22:21Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.7", GitCommit:"dd5e1a2978fd0b97d9b78e1564398aeea7e7fe92", GitTreeState:"clean", BuildDate:"2018-04-18T23:58:35Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

这是用来设置网络的：

kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=10.132.113.68 --kubernetes-version stable-1.8

所有 pods 启动成功：

# kubectl get pods --all-namespaces
NAMESPACE     NAME                                                        READY     STATUS      RESTARTS   AGE
default       datadog-agent-5cht2                                         1/1       Running     0          12d
default       datadog-agent-5r7rw                                         1/1       Running     0          12d
default       datadog-agent-b7t5q                                         1/1       Running     0          12d
default       vizix-services-7bdccb48c4-2q8js                             1/1       Running     0          19m
default       vizix-tools-cpr88                                           0/1       Completed   0          12d
kube-system   etcd-kubctl-s-2vcpu-4gb-nyc3-01-master                      1/1       Running     0          27d
kube-system   kube-apiserver-kubctl-s-2vcpu-4gb-nyc3-01-master            1/1       Running     0          27d
kube-system   kube-controller-manager-kubctl-s-2vcpu-4gb-nyc3-01-master   1/1       Running     0          27d
kube-system   kube-dns-6f4fd4bdf-f7ssn                                    3/3       Running     0          27d
kube-system   kube-flannel-ds-dm5w4                                       1/1       Running     0          27d
kube-system   kube-flannel-ds-ns58w                                       1/1       Running     0          27d
kube-system   kube-flannel-ds-prnvf                                       1/1       Running     1          27d
kube-system   kube-flannel-ds-xck8p                                       1/1       Running     0          27d
kube-system   kube-proxy-2xrhl                                            1/1       Running     0          27d
kube-system   kube-proxy-lnt9r                                            1/1       Running     0          27d
kube-system   kube-proxy-m74ms                                            1/1       Running     0          27d
kube-system   kube-proxy-vqdxt                                            1/1       Running     0          27d
kube-system   kube-scheduler-kubctl-s-2vcpu-4gb-nyc3-01-master            1/1       Running     0          27d
kube-system   kubernetes-dashboard-5bd6f767c7-7qp75                       1/1       Running     0          26d

需要连接Kafka的POD使用PING或者Telnet就可以到达集群：

# kubectl exec -it vizix-services-7bdccb48c4-2q8js bash
bash-4.2# ping 10.132.123.177
PING 10.132.123.177 (10.132.123.177) 56(84) bytes of data.
64 bytes from 10.132.123.177: icmp_seq=1 ttl=63 time=0.540 ms
64 bytes from 10.132.123.177: icmp_seq=2 ttl=63 time=0.518 ms
64 bytes from 10.132.123.177: icmp_seq=3 ttl=63 time=0.432 ms
64 bytes from 10.132.123.177: icmp_seq=4 ttl=63 time=0.527 ms
^C
--- 10.132.123.177 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.432/0.504/0.540/0.045 ms
bash-4.2# telnet 10.132.123.177 9092
Trying 10.132.123.177...
Connected to 10.132.123.177.
Escape character is '^]'.
^CConnection closed by foreign host.
bash-4.2#

但是 JAVA 应用程序不能。只使用docker，可以正常连接，但是当Kubernetes启动pod时，就连接不上了。

2018-05-30 01:25:06,993+0000 WARN  [localhost-startStop-1] com.tierconnect.riot.commons.services.broker.KafkaPublisher:: - 
Check if exists a connection to kafka server 10.132.123.177:9092 and services is able to publish to kafka.

Kubernetes 有什么东西可以阻止某些特定的应用程序协议从一个节点连接到外部主机吗？

这是服务的部署 YAML 文件：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: microservice1
  labels:
    app: services
spec:
  replicas: 1
  selector:
    matchLabels:
      app: services
  template:
    metadata:
      labels:
        app: services
    spec:
      containers:
      - name: microservice1
        image: random/java-image:v6.5.2
        env:
      - name: KAFKA_SERVERS
        value: "10.132.123.177:9092"

Answer 1

我使用端点和服务解决了这个问题。这样连接由 Kubernetes 管理：

    ---
    kind: "Service"
    apiVersion: "v1"
    metadata:
      name: "kafka"
    spec:
      ports:
        -
          name: "kafka"
          protocol: "TCP"
          port: 9092
          targetPort: 9092
          nodePort: 0
    ---
    kind: "Endpoints"
    apiVersion: "v1"
    metadata:
      name: "kafka"
    subsets:
      -
        addresses:
          -
            ip: "10.128.0.2"
        ports:
          -
            port: 9092
            name: "kafka"

Kubernetes 节点无法访问外部 Kafka 集群

Kubernetes node can't reach external Kafka Cluster

docker

kubernetes

flannel