使用 Kubernetes 在 OrientDB 集群中未发现节点

Question

我使用 this guide 在 Kubernetes 中设置 OrientDB 集群。但是，似乎每个 pod 上的每个节点都创建了自己的集群，而不是加入共享集群。所以每个 pod 上的日志显示这样一条消息：

Members [1] {
    Member [pod-ip]:5701 - generated id
}

什么会导致这样的问题？

我的 orientdb-server-config 文件看起来是这样的：

<handler class="com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin">
     <parameters>
          <parameter value="true" name="enabled"/>
          <parameter value="orientdb/config/default-distributed-db-config.json" name="configuration.db.default"/>
          <parameter value="orientdb/config/hazelcast.xml" name="configuration.hazelcast"/>
          <parameter name="nodeName" value="$pod_dns" />
     </parameters>
</handler>

我的 hazelcast.xml 文件如下所示（pod_dns 是存储在 env 中的 pod 的名称）：

 </network>
    <properties>
        <property name="hazelcast.discovery.enabled">true</property>
    </properties>
    <network>
        <join>
            <multicast enabled="false"/>
            <tcp-ip enabled="false"/>
            <discovery-strategies>
                <discovery-strategy enabled="true"
                                    class="com.hazelcast.kubernetes.HazelcastKubernetesDiscoveryStrategy">
                    <properties>
                        <property name="service-dns">pod_dns.default.svc.cluster.local</property>
                        <property name="service-dns-timeout">10</property>
                    </properties>
                </discovery-strategy>
            </discovery-strategies>
        </join>
    </network>

Kubernetes 状态集。 Bash 挂载并执行 hazelcast 和 orientdb-server-config 文件的脚本（用于根据每个 pod 上的 env 值设置更新）：

kind: StatefulSet
apiVersion: apps/v1
metadata:
  name: orientdbservice
spec:
  serviceName: orientdbservice
  replicas: 3
  podManagementPolicy: Parallel
  selector:
    matchLabels:
      service: orientdb
      type: container-deployment
  template:
    metadata:
      labels:
        service: orientdb
        type: container-deployment
    spec:
      containers:
      - name: orientdbservice
        image: orientdb:2.2.36
        command: ["/bin/sh","-c", " cp /configs/* /orientdb/config/ ; chmod +x /orientdb/config/hazelcast_template.sh ; chmod +x /orientdb/config/server_config_template.sh ; sh /orientdb/config/hazelcast_template.sh ; sh /orientdb/config/server_config_template.sh ; /orientdb/bin/server.sh -Ddistributed=true" ]
        env:
        - name: ORIENTDB_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: orientdb-password
              key: password.txt
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        ports:
        - containerPort: 2424
          name: port-binary
        - containerPort: 2480
          name: port-http
        - containerPort: 5701
          name: hazelcast
        volumeMounts:
        - name: config
          mountPath: /orientdb/config
        - name: orientdb-config-template-hazelcast
          mountPath: /configs/hazelcast_template.sh
          subPath: hazelcast_template.sh
        - name: orientdb-config-template-server
          mountPath: /configs/server_config_template.sh
          subPath: server_config_template.sh
        - name: orientdb-config-distributed
          mountPath: /configs/default-distributed-db-config.json
          subPath: default-distributed-db-config.json
        - name: orientdb-databases
          mountPath: /orientdb/databases
        - name: orientdb-backup
          mountPath: /orientdb/backup
      volumes:
      - name: config
        emptyDir: {}
      - name: orientdb-config-template-hazelcast
        configMap:
          name: orientdb-configmap-template-hazelcast
      - name: orientdb-config-template-server
        configMap:
          name: orientdb-configmap-template-server
      - name: orientdb-config-distributed
        configMap:
          name: orientdb-configmap-distributed
  volumeClaimTemplates:
  - metadata:
      name: orientdb-databases
      labels:
        service: orientdb
        type: pv-claim
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 20Gi
  - metadata:
      name: orientdb-backup
      labels:
        service: orientdb
        type: pv-claim
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 10Gi

Answer 1

问题出在 Hazelcast0Kubernetes 插件配置上。首先，有必要将 OrientDB 版本更新到最新版本 - 3.0.10，其中嵌入了最新的 Hazelcast 版本。此外，我已将 hazelcast-kubernetes.jar 依赖文件直接安装到 /orientdb/lib 文件夹中，它开始正常工作。问题不在于配置文件，而在于 OrientDB 的依赖设置。

使用 Kubernetes 在 OrientDB 集群中未发现节点

Nodes are not discovered in OrientDB cluster with Kubernetes

service-discovery

orientdb

kubernetes