kafka + This server is not the leader for that topic-partition + kafka 生产者

kafka + This server is not the leader for that topic-partition + kafka producer

我们从每台 3 kafka 机器(我们在集群中有 3 kafka)server.log 得到很多错误

[2019-12-05 13:25:09,529] ERROR [ReplicaFetcherThread-0-1], Error for partition [jdty.dee.rules.time,91] to broker 1001:org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition. (kafka.server.ReplicaFetcherThread)
[2019-12-05 13:25:09,529] ERROR [ReplicaFetcherThread-0-1], Error for partition [jdty.dee.export.profiles,96] to broker 1001:org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition. (kafka.server.ReplicaFetcherThread)
[2019-12-05 13:25:09,529] ERROR [ReplicaFetcherThread-0-1], Error for partition [jdty.dee.control.tt.state,40] to broker 1001:org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition. (kafka.server.ReplicaFetcherThread)
[2019-12-05 13:25:09,529] ERROR [ReplicaFetcherThread-0-1], Error for partition [jdty.dee.control.tt,67] to broker 1001:org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition. (kafka.server.ReplicaFetcherThread)

根据我的理解,每个主题由一个或多个经纪人提供服务 - 一个是领导者,其余经纪人是追随者。

生产者需要向领导 Broker 发送新消息,领导 Broker 在内部将数据复制给所有追随者。

我假设,您的生产者客户端没有连接到正确的 Broker,它连接到一个跟随者而不是领导者,并且这个跟随者拒绝了您的发送请求。

所以我的问题是 - 如何配置生产者配置以避免此类错误?

Follower broker 从 leader broker 获取(leader 没有推送给 followers)。因此,似乎跟随者经纪人试图从错误的(领导者)经纪人那里获取。这可能发生在分区的领导者改变时。相应的 follower broker 应该自动更新其集群元数据以重新发现新的 leader。如果错误仍然存​​在,则表示此跟随者经纪人在更新其元数据时遇到问题。