无法通过 AWS EKS 中的 LB 连接到 Apache Ignite (jdbc)

Question

我正在尝试连接到部署在 Kubernetes 集群上的 Apache Ignite (2.8.0)。集群本身看起来很好，因为我能够执行 pods 并通过 sqlline 工具访问，如下所示

root@ip-172-17-28-68:/opt/apache-ignite-2.8.0-bin# kubectl exec -it ignite-cluster-6d69696b67-8vvmm /bin/bash

bash-4.4# apache-ignite/bin/sqlline.sh --verbose=true -u jdbc:ignite:thin://127.0.0.1:10800/
issuing: !connect jdbc:ignite:thin://127.0.0.1:10800/ '' '' org.apache.ignite.IgniteJdbcThinDriver
Connecting to jdbc:ignite:thin://127.0.0.1:10800/
Connected to: Apache Ignite (version 2.8.0#20200226-sha1:341b01df)
Driver: Apache Ignite Thin JDBC Driver (version 2.8.0#20200226-sha1:341b01df)
Autocommit status: true
Transaction isolation: TRANSACTION_REPEATABLE_READ
sqlline version 1.3.0
0: jdbc:ignite:thin://127.0.0.1:10800/>

但是，如果我尝试从外部 LB 连接，则会出现以下错误。

root@ip-172-17-28-68:/opt/apache-ignite-2.8.0-bin/bin# ./sqlline.sh --verbose=true -u jdbc:ignite:thin://abc-123.us-east-1.elb.amazonaws.com:10800
issuing: !connect jdbc:ignite:thin://abc-123.us-east-1.elb.amazonaws.com:10800 '' '' org.apache.ignite.IgniteJdbcThinDriver
Connecting to jdbc:ignite:thin://abc-123.us-east-1.elb.amazonaws.com:10800
Error: Failed to connect to server [url=jdbc:ignite:thin://weiury734ry34ry34urt.us-east-1.elb.amazonaws.com:10800/PUBLIC] (state=08001,code=0)
java.sql.SQLException: Failed to connect to server [url=jdbc:ignite:thin://abc123.us-east-1.elb.amazonaws.com:10800/PUBLIC]
        at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.handleConnectExceptions(JdbcThinConnection.java:1529)
        at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.connectInCommonMode(JdbcThinConnection.java:1506)
        at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.ensureConnected(JdbcThinConnection.java:231)
        at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.<init>(JdbcThinConnection.java:210)
        at org.apache.ignite.IgniteJdbcThinDriver.connect(IgniteJdbcThinDriver.java:154)
        at sqlline.DatabaseConnection.connect(DatabaseConnection.java:156)
        at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:204)
        at sqlline.Commands.connect(Commands.java:1095)
        at sqlline.Commands.connect(Commands.java:1001)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
        at sqlline.SqlLine.dispatch(SqlLine.java:791)
        at sqlline.SqlLine.initArgs(SqlLine.java:566)
        at sqlline.SqlLine.begin(SqlLine.java:643)
        at sqlline.SqlLine.start(SqlLine.java:373)
        at sqlline.SqlLine.main(SqlLine.java:265)
        Suppressed: java.io.IOException: Failed to read incoming message (not enough data).
                at org.apache.ignite.internal.jdbc.thin.JdbcThinTcpIo.read(JdbcThinTcpIo.java:546)
                at org.apache.ignite.internal.jdbc.thin.JdbcThinTcpIo.read(JdbcThinTcpIo.java:524)
                at org.apache.ignite.internal.jdbc.thin.JdbcThinTcpIo.handshake(JdbcThinTcpIo.java:266)
                at org.apache.ignite.internal.jdbc.thin.JdbcThinTcpIo.<init>(JdbcThinTcpIo.java:212)
                at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.connectInCommonMode(JdbcThinConnection.java:1477)
                ... 17 more
        Suppressed: java.io.IOException: Failed to read incoming message (not enough data).
                at org.apache.ignite.internal.jdbc.thin.JdbcThinTcpIo.read(JdbcThinTcpIo.java:546)
                at org.apache.ignite.internal.jdbc.thin.JdbcThinTcpIo.read(JdbcThinTcpIo.java:524)
                at org.apache.ignite.internal.jdbc.thin.JdbcThinTcpIo.handshake(JdbcThinTcpIo.java:266)
                at org.apache.ignite.internal.jdbc.thin.JdbcThinTcpIo.<init>(JdbcThinTcpIo.java:212)
                at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.connectInCommonMode(JdbcThinConnection.java:1477)
                ... 17 more
sqlline version 1.3.0

有趣的是，通过 LB telnet 到端口表示已连接，如下所示。

root@ip-172-17-28-68:/opt/apache-ignite-2.8.0-bin/bin# telnet abc-123.us-east-1.elb.amazonaws.com 10800
Trying 3.209.73.243...
Connected to abc-123.us-east-1.elb.amazonaws.com.
Escape character is '^]'.
Connection closed by foreign host.

为什么即使 telnet 在端口 10800 上工作，也无法通过 jdbc 客户端连接？ 我也尝试通过DBeaver连接，但错误是一样的。

以下是 Kubernetes EKS 部署中使用的服务和部署 yaml。

点燃-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: ignite
  namespace: default
  labels:
    app: ignite
spec:
  type: LoadBalancer
  ports:
    - name: rest
      port: 8080
      protocol: TCP
      targetPort: 8080
    - name: sql
      port: 10800
      protocol: TCP
      targetPort: 10800
    - name: thinclients
      port: 10900
      protocol: TCP
      targetPort: 10900
  selector:
    app: ignite

点燃-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ignite-cluster
  namespace: default
spec:
  selector:
    matchLabels:
      app: ignite
  replicas: 2
  template:
    metadata:
      labels:
        app: ignite
    spec:
      serviceAccount: ignite
      containers:
      - name: ignite-node
        image: apacheignite/ignite:2.8.0
        env:
        - name: OPTION_LIBS
          value: ignite-kubernetes
        - name: CONFIG_URI
          value: https://raw.githubusercontent.com/apache/ignite/master/modules/kubernetes/config/example-kube.xml
        ports:
        - containerPort: 11211 # REST port number.
        - containerPort: 47100 # communication SPI port number.
        - containerPort: 47500 # discovery SPI port number.
        - containerPort: 49112 # JMX port number.
        - containerPort: 10800 # SQL port number.

Answer 1

检查服务本身：kubectl describe services my-service

https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#finding-your-ip-address

确保外部 IP 已提供且有效。

将 K8 限制为一个 pod，然后重试。

设置externalTrafficPolicy和pod选择器

externalTrafficPolicy: Local
  selector:
    statefulset.kubernetes.io/pod-name: pod-name

https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip

https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-name-label

瘦客户端需要与特定的 pod/server

通信

检查服务器日志以确保数据在流动。

无法通过 AWS EKS 中的 LB 连接到 Apache Ignite (jdbc)

Unable to connect to Apache Ignite (jdbc) through LB in AWS EKS

jdbc

kubernetes

ignite

amazon-eks