Kafka Connect - 获取具有相关 ID UNKNOWN_TOPIC_OR_PARTITION 的元数据时出错
Kafka Connect - Error while fetching metadata with correlation id UNKNOWN_TOPIC_OR_PARTITION
我们有 Kafka Connect 运行 一个 postgres 连接器，它从数据库中提取更改并将它们放入主题中。我们遇到错误
ERROR WorkerSourceTask{id=wem-postgres-source-0} Failed to flush, timed out while waiting for producer to flush outstanding 1 messages
接着是
ERROR WorkerSourceTask{id=wem-postgres-source-0} Failed to commit offsets (org.apache.kafka.connect.runtime.SourceTaskOffsetCommitter)
接着是
[2020-01-30 09:51:52,219] WARN [Producer clientId=producer-8] Error while fetching metadata with correlation id 606994 : {topicname=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
我们已验证该主题确实存在，对该主题进行描述给我们这个
zookeeper-1 [root@XX /bin]# ./kafka-topics --describe --zookeeper <zookeeper>:2181 --topic topicname
Topic:topicname PartitionCount:1 ReplicationFactor:3 Configs:
Topic: topicname Partition: 0 Leader: 2 Replicas: 2,3,1 Isr: 3,2,1
ERROR WorkerSourceTask{id=wem-postgres-source-0} Failed to flush, timed out while waiting for producer to flush outstanding 1 messages
和
[2020-01-30 09:51:52,219] WARN [Producer clientId=producer-8] Error while fetching metadata with correlation id 606994 : {topicname=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
来自同一期。您是否尝试过访问您的 Kafka 集群或升级您的 Kafka？最新的 kafka-topics 脚本使用代理主机而不是 ZooKeeper 主机来描述主题。
至于
ERROR WorkerSourceTask{id=wem-postgres-source-0} Failed to commit offsets (org.apache.kafka.connect.runtime.SourceTaskOffsetCommitter)
确保您的 Kafka Connect offset.storage.topic 已经在 Kafka 中创建。工作人员将偏移量冲入此主题。
我们有 Kafka Connect 运行 一个 postgres 连接器，它从数据库中提取更改并将它们放入主题中。我们遇到错误
ERROR WorkerSourceTask{id=wem-postgres-source-0} Failed to flush, timed out while waiting for producer to flush outstanding 1 messages
接着是
ERROR WorkerSourceTask{id=wem-postgres-source-0} Failed to commit offsets (org.apache.kafka.connect.runtime.SourceTaskOffsetCommitter)
接着是
[2020-01-30 09:51:52,219] WARN [Producer clientId=producer-8] Error while fetching metadata with correlation id 606994 : {topicname=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
我们已验证该主题确实存在，对该主题进行描述给我们这个
zookeeper-1 [root@XX /bin]# ./kafka-topics --describe --zookeeper <zookeeper>:2181 --topic topicname
Topic:topicname PartitionCount:1 ReplicationFactor:3 Configs:
Topic: topicname Partition: 0 Leader: 2 Replicas: 2,3,1 Isr: 3,2,1
ERROR WorkerSourceTask{id=wem-postgres-source-0} Failed to flush, timed out while waiting for producer to flush outstanding 1 messages
和
[2020-01-30 09:51:52,219] WARN [Producer clientId=producer-8] Error while fetching metadata with correlation id 606994 : {topicname=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
来自同一期。您是否尝试过访问您的 Kafka 集群或升级您的 Kafka？最新的 kafka-topics 脚本使用代理主机而不是 ZooKeeper 主机来描述主题。
至于
ERROR WorkerSourceTask{id=wem-postgres-source-0} Failed to commit offsets (org.apache.kafka.connect.runtime.SourceTaskOffsetCommitter)
确保您的 Kafka Connect offset.storage.topic 已经在 Kafka 中创建。工作人员将偏移量冲入此主题。