Strimzi Mirrormaker2 和事件中心奇怪的错误，无法查明

Question

我正在尝试以一种方式将 Kafka 同步到 Azure 事件中心。我遵循了我能找到的所有教程，但都无济于事。似乎没有任何效果，因为我不断收到模糊的错误。下面是用于部署的配置。我们甚至使用 RootManageSharedAccessKey 来确保没有任何阻塞。我有 Kafka，KafkaConnect 部署没有问题。

MirrorMaker2 在 Kafka 和 Kafka 之间工作，但是当我尝试与 EventHub 同步时没有骰子。

为了测试9093端口是否可以访问，我使用telnet访问成功

MM2 配置

cat <<EOF | kubectl apply -n kafka-cloud -f -
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaMirrorMaker2
metadata:
  name: mirror-maker-eventhub
spec:
  version: 3.1.0
  replicas: 1
  connectCluster: "eventhub"
  clusters:
  - alias: "my-kafka-cluster"
    bootstrapServers: my-kafka-cluster-kafka-bootstrap:9092
  - alias: "eventhub"
    bootstrapServers: XXXXXXXXXXXXXXXXXXXX.servicebus.windows.net:9093
    config:
      config.storage.replication.factor: 1
      offset.storage.replication.factor: 1
      status.storage.replication.factor: 1
      producer.connections.max.idle.ms: 180000
      producer.metadata.max.age.ms: 180000
      security.protocol: SASL_SSL
      sasl.mechanism: PLAIN
      sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="Endpoint=sb://XXXXXXXXXXXXXXXXXXXX.servicebus.windows.net/;SharedAccessKeyName=RootKeyAccess;SharedAccessKey=XXXXXXXXXXXXXXXXXXXX";
    tls:
      trustedCertificates: []
  mirrors:
  - sourceCluster: "my-kafka-cluster"
    targetCluster: "eventhub"
    sourceConnector:
      config:
        replication.factor: 1
        offset-syncs.topic.replication.factor: 1
        sync.topic.acls.enabled: "false"
    heartbeatConnector:
      config:
        heartbeats.topic.replication.factor: 1
    checkpointConnector:
      config:
        checkpoints.topic.replication.factor: 1
    topicsPattern: ".*"
    groupsPattern: ".*"
EOF

第一个问题是这些警告。我知道这只是信息，但部署一直在崩溃，不会保持运行ning。我也知道 Kafka 这些类型的日志充其量是模糊的。

2022-03-15 21:21:06,151 INFO [AdminClient clientId=adminclient-1] Node -1 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-1]

2022-03-15 21:21:06,152 INFO [AdminClient clientId=adminclient-1] Cancelled in-flight METADATA request with correlation id 59 due to node -1 being disconnected (elapsed time since creation: 87ms, elapsed time since send: 87ms, request timeout: 16401ms) (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-1]

然后我在日志中发现了这个错误。我不知道问题出在哪里，也不知道如何解决。

2022-03-15 21:22:55,572 INFO [AdminClient clientId=adminclient-1] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager) [kafka-admin-client-thread | adminclient-1]

>> org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata

2022-03-15 21:22:55,574 INFO App info kafka.admin.client for adminclient-1 unregistered (org.apache.kafka.common.utils.AppInfoParser) [kafka-admin-client-thread | adminclient-1]
2022-03-15 21:22:55,575 INFO [AdminClient clientId=adminclient-1] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager) [kafka-admin-client-thread | adminclient-1]
org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata
2022-03-15 21:22:55,575 INFO [AdminClient clientId=adminclient-1] Timed out 1 remaining operation(s) during close. (org.apache.kafka.clients.admin.KafkaAdminClient) [kafka-admin-client-thread | adminclient-1]
2022-03-15 21:22:55,582 INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics) [kafka-admin-client-thread | adminclient-1]
2022-03-15 21:22:55,582 INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics) [kafka-admin-client-thread | adminclient-1]
2022-03-15 21:22:55,582 INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics) [kafka-admin-client-thread | adminclient-1]
2022-03-15 21:22:55,583 ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectDistributed) [main]

>> org.apache.kafka.connect.errors.ConnectException: Failed to connect to and describe Kafka cluster. Check workers broker connection and security properties.

at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:70)
at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:51)
at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:97)
at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:80)

>> Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: listNodes

当我运行描述时出现此错误

Status:
  Conditions:
    Last Transition Time:  2022-03-15T21:25:49.149589Z
    Message:               Exceeded timeout of 300000ms while waiting for Deployment resource mirror-maker-eventhub-mirrormaker2 in namespace kafka-cloud to be ready
    Reason:                TimeoutException
    Status:                True
    Type:                  NotReady
  Label Selector:          strimzi.io/cluster=mirror-maker-eventhub,strimzi.io/name=mirror-maker-eventhub-mirrormaker2,strimzi.io/kind=KafkaMirrorMaker2
  Observed Generation:     1
  Replicas:                1
  URL:                     http://mirror-maker-eventhub-mirrormaker2-api.kafka-cloud.svc:8083
Events:                    <none>

当我在mm2中添加认证块时：

    authentication: 
      type: plain 
      username: $ConnectionString
      passwordSecret: 
        secretName: eventhubssecret 
        password: eventhubspassword

我收到此错误，即使秘密存在并经过验证。

Status:
  Conditions:
    Last Transition Time:  2022-03-15T23:30:31.200105Z
    Message:               PLAIN authentication selected, but username or password configuration is missing.
    Reason:                InvalidResourceException
    Status:                True
    Type:                  NotReady
  Observed Generation:     2
  Replicas:                0
Events:                    <none>

如果有人有任何想法，将不胜感激。折腾了一整天也无济于事。我删除了 using secrets，因为它导致未提供名称和密码的错误。很奇怪。

谢谢。

Answer 1

如果没有完整的日志，我不确定这是您唯一的问题。但是您的身份验证配置肯定是错误的。您需要在 authentiction 部分配置它：

    authentication:
      type: plain
      username: $ConnectionString
      passwordSecret:
        secretName: eventhubssecret
        password: eventhubspassword

您可以查看文档以获取有关如何配置身份验证的更多信息（例如 here or here). You can also check this blog post，它专门介绍如何将 MM2 与 EventHub 结合使用。它有点旧，但也展示了如何进行身份验证。

Answer 2

我没有意识到 git 链接到这里，所以我将问题移至

https://github.com/strimzi/strimzi-kafka-operator/discussions/6526

所有跟进和希望解决方案将在该线程中。仍然无法访问天蓝色。那边会跟进的。

Answer 3

已找到答案。

问题是使用 cat <

我认为这是因为用户名中的 $。 EH 需要这个作为连接的实际用户名。一旦我将上面的内容变成 cat <<EOF and the last EOF 之间的文件，它就从 CLI 运行不做任何更改。

成功了。

kubectl apply -n kafka-cloud -f fileName.yaml

使用 EH 时使用文件。不要 cat <

Strimzi Mirrormaker2 和事件中心奇怪的错误，无法查明

Strimzi Mirrormaker2 and Event Hub Odd Error, Can't Pinpoint

apache-kafka

kubernetes

azure-eventhub

apache-kafka-connect

strimzi

MM2 配置