Kafka Consumer 标记协调器 2147483647 已死
Kafka Consumer Marking the coordinator 2147483647 dead
我正在使用 Kafka Server 0.9 以及消费者 kafka-client 版本 0.9 和 kafka-producer 0.8.2。
一切都很好，除了我收到很多信息表明协调器已死于消费者
2016-02-25 19:30:45.046 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.048 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.049 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.050 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.051 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.052 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.053 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.054 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.055 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.056 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.057 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.058 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.059 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.060 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.061 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.062 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.062 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.063 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.064 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.065 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.066 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.067 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.068 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.068 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.069 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.070 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.071 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.072 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.072 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.073 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.074 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.075 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.075 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.076 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.077 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.078 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.079 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.079 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.080 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.081 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.082 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.083 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.083 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.084 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.085 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.086 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.086 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.087 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.088 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.089 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.089 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.090 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.091 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.093 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.094 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.094 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
我还注意到生产者每 10 分钟断开一次连接，如下所示
2016-03-12 15:55:36 INFO [pool-1-thread-1] - Fetching metadata from broker id:0,host:192.168.72.30,port:9092 with correlation id 41675 for 1 topic(s) Set(act)
2016-03-12 15:55:36 INFO [pool-1-thread-1] - Connected to 192.168.72.30:9092 for producing
2016-03-12 15:55:36 INFO [pool-1-thread-1] - Disconnecting from 192.168.72.30:9092
2016-03-12 15:55:36 INFO [pool-1-thread-1] - Disconnecting from kafkauk.XXXXXXXXXX.co:9092
2016-03-12 15:55:36 INFO [pool-1-thread-1] - Connected to kafkauk.XXXXXXXXXX.co:9092 for producing
这是我的生产者配置
metadata.broker.list=192.168.72.30:9092
serializer.class=kafka.serializer.StringEncoder
request.required.acks=1
linger.ms=2000
batch.size=500
和消费者配置
bootstrap.servers: kafkauk.xxxxxxxx.co:9092
group.id: cdrServer
client.id: cdrServer
enable.auto.commit: true
auto.commit.interval.ms: 1000
session.timeout.ms: 30000
key.deserializer: org.apache.kafka.common.serialization.StringDeserializer
value.deserializer: org.apache.kafka.common.serialization.StringDeserializer
我不明白这些是什么意思，我应该忽略它们还是我在配置中遗漏了一些东西
在我将 kafka 更改为消费者的调试级别后，我发现了以下
2016-03-13 18:21:55.586 DEBUG 5469 --- [ cdrServer] org.apache.kafka.clients.NetworkClient : Node 2147483647 disconnected.
2016-03-13 18:21:55.586 INFO 5469 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-03-13 18:21:55.586 DEBUG 5469 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Issuing group metadata request to broker 0
2016-03-13 18:21:55.586 DEBUG 5469 --- [ cdrServer] org.apache.kafka.clients.NetworkClient : Sending metadata request ClientRequest(expectResponse=true
, callback=null, request=RequestSend(header={api_key=3,api_version=0,correlation_id=183025,client_id=cdrServer}, body={topics=[act]}), isInitiatedByNetworkCli
ent, createdTimeMs=1457893315586, sendTimeMs=0) to node 0
2016-03-13 18:21:55.591 DEBUG 5469 --- [ cdrServer] org.apache.kafka.clients.Metadata : Updated cluster metadata version 296 to Cluster(nodes = [N
ode(0, kafkauk.xxxxxxxxx.co, 9092)], partitions = [Partition(topic = act, partition = 0, leader = 0, replicas = [0,], isr = [0,]])
2016-03-13 18:21:55.592 DEBUG 5469 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Group metadata response ClientResponse(receivedTimeMs=1457
893315592, disconnected=false, request=ClientRequest(expectResponse=true, callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFu
tureCompletionHandler@1e2de777, request=RequestSend(header={api_key=10,api_version=0,correlation_id=183024,client_id=cdrServer}, body={group_id=cdrServer}), c
reatedTimeMs=1457893315586, sendTimeMs=1457893315586), responseBody={error_code=0,coordinator={node_id=0,host=kafkauk.xxxxxxxx.co,port=9092}})
我不确定这是网络问题，因为它恰好每 9 分钟发生一次
更新
我发现这与
直接相关
connections.max.idle.ms: 300000
无论我输入什么，我都会在这个值时断开连接
当 Consumer Client 和 Coordinator 之间存在网络通信错误时，会将协调器标记为已死亡（这也可能发生在 Coordinator 死亡且组需要重新平衡时）。有多种情况（偏移量提交请求、获取偏移量等）会导致此问题。我会建议您研究导致这种情况的原因
在我的例子中，当我尝试手动分配分区时，消息出现在日志中。在我阅读 api 新消费者的文档后，请遵循通知：
It is also possible for the consumer to manually assign specific partitions (similar to the older "simple" consumer) using assign(Collection). In this case, dynamic partition assignment and consumer group coordination will be disabled.
也就是说，如果您有这样的代码：
KafkaConsumer<String, String> consumer = new KafkaConsumer(props);
consumer.assign( Arrays.asList(
new TopicPartition("topic", 0),
new TopicPartition("topic", 1)
));
然后消息 "Marking the coordinator 2147483647 dead" 将始终放入我们的日志中。
这基本上是你无法到达Kafka。
在我的例子中，我是 运行 Kafka in vagrant box，如果我启动 VPN，它会刷新
vagrant ip，因此它无法连接到它。
可能的解决方案： 在这种情况下，请停止 VPN 并启动您的 vagrant。
我今天遇到这个问题并解决了它（暂时，我可以补充一下）。我已经发布了关于我是如何做到的答案 here。
我遇到过同样的问题。最后在遵循 Shannon 关于 TRACING 日志的建议之后，我使用了：
logging.level.org.apache.kafka=TRACE
发现我的客户试图将 Euler:9092 解析为协调器...本地名称！！
所以我注释掉并更改了 server.properties 文件中的 listeners 和 advertised.listeners 值。
它现在正在工作！ :-)
这也可能与较长的垃圾收集停止世界阶段有关。在我的例子中，我在 > 10 秒的 GC 之后遇到了这条消息。
这个错误多发生在coordinator和consumer之间有冲突的时候。你应该做的第一件事是在 server.properties 中暴露监听端口，其次你需要删除 kafka-logs 下的所有日志。不要忘记在这些步骤之后重新启动服务器和 zookeeper。它将解决问题。
我正在使用 Kafka Server 0.9 以及消费者 kafka-client 版本 0.9 和 kafka-producer 0.8.2。
一切都很好，除了我收到很多信息表明协调器已死于消费者
2016-02-25 19:30:45.046 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.048 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.049 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.050 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.051 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.052 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.053 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.054 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.055 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.056 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.057 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.058 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.059 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.060 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.061 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.062 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.062 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.063 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.064 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.065 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.066 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.067 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.068 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.068 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.069 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.070 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.071 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.072 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.072 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.073 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.074 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.075 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.075 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.076 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.077 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.078 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.079 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.079 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.080 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.081 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.082 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.083 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.083 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.084 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.085 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.086 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.086 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.087 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.088 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.089 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.089 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.090 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.091 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.093 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.094 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-02-25 19:30:45.094 INFO 10263 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
我还注意到生产者每 10 分钟断开一次连接，如下所示
2016-03-12 15:55:36 INFO [pool-1-thread-1] - Fetching metadata from broker id:0,host:192.168.72.30,port:9092 with correlation id 41675 for 1 topic(s) Set(act)
2016-03-12 15:55:36 INFO [pool-1-thread-1] - Connected to 192.168.72.30:9092 for producing
2016-03-12 15:55:36 INFO [pool-1-thread-1] - Disconnecting from 192.168.72.30:9092
2016-03-12 15:55:36 INFO [pool-1-thread-1] - Disconnecting from kafkauk.XXXXXXXXXX.co:9092
2016-03-12 15:55:36 INFO [pool-1-thread-1] - Connected to kafkauk.XXXXXXXXXX.co:9092 for producing
这是我的生产者配置
metadata.broker.list=192.168.72.30:9092
serializer.class=kafka.serializer.StringEncoder
request.required.acks=1
linger.ms=2000
batch.size=500
和消费者配置
bootstrap.servers: kafkauk.xxxxxxxx.co:9092
group.id: cdrServer
client.id: cdrServer
enable.auto.commit: true
auto.commit.interval.ms: 1000
session.timeout.ms: 30000
key.deserializer: org.apache.kafka.common.serialization.StringDeserializer
value.deserializer: org.apache.kafka.common.serialization.StringDeserializer
我不明白这些是什么意思，我应该忽略它们还是我在配置中遗漏了一些东西
在我将 kafka 更改为消费者的调试级别后，我发现了以下
2016-03-13 18:21:55.586 DEBUG 5469 --- [ cdrServer] org.apache.kafka.clients.NetworkClient : Node 2147483647 disconnected.
2016-03-13 18:21:55.586 INFO 5469 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Marking the coordinator 2147483647 dead.
2016-03-13 18:21:55.586 DEBUG 5469 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Issuing group metadata request to broker 0
2016-03-13 18:21:55.586 DEBUG 5469 --- [ cdrServer] org.apache.kafka.clients.NetworkClient : Sending metadata request ClientRequest(expectResponse=true
, callback=null, request=RequestSend(header={api_key=3,api_version=0,correlation_id=183025,client_id=cdrServer}, body={topics=[act]}), isInitiatedByNetworkCli
ent, createdTimeMs=1457893315586, sendTimeMs=0) to node 0
2016-03-13 18:21:55.591 DEBUG 5469 --- [ cdrServer] org.apache.kafka.clients.Metadata : Updated cluster metadata version 296 to Cluster(nodes = [N
ode(0, kafkauk.xxxxxxxxx.co, 9092)], partitions = [Partition(topic = act, partition = 0, leader = 0, replicas = [0,], isr = [0,]])
2016-03-13 18:21:55.592 DEBUG 5469 --- [ cdrServer] o.a.k.c.c.internals.AbstractCoordinator : Group metadata response ClientResponse(receivedTimeMs=1457
893315592, disconnected=false, request=ClientRequest(expectResponse=true, callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFu
tureCompletionHandler@1e2de777, request=RequestSend(header={api_key=10,api_version=0,correlation_id=183024,client_id=cdrServer}, body={group_id=cdrServer}), c
reatedTimeMs=1457893315586, sendTimeMs=1457893315586), responseBody={error_code=0,coordinator={node_id=0,host=kafkauk.xxxxxxxx.co,port=9092}})
我不确定这是网络问题，因为它恰好每 9 分钟发生一次
更新
我发现这与
直接相关connections.max.idle.ms: 300000
无论我输入什么，我都会在这个值时断开连接
当 Consumer Client 和 Coordinator 之间存在网络通信错误时，会将协调器标记为已死亡（这也可能发生在 Coordinator 死亡且组需要重新平衡时）。有多种情况（偏移量提交请求、获取偏移量等）会导致此问题。我会建议您研究导致这种情况的原因
在我的例子中，当我尝试手动分配分区时，消息出现在日志中。在我阅读 api 新消费者的文档后，请遵循通知：
It is also possible for the consumer to manually assign specific partitions (similar to the older "simple" consumer) using assign(Collection). In this case, dynamic partition assignment and consumer group coordination will be disabled.
也就是说，如果您有这样的代码：
KafkaConsumer<String, String> consumer = new KafkaConsumer(props);
consumer.assign( Arrays.asList(
new TopicPartition("topic", 0),
new TopicPartition("topic", 1)
));
然后消息 "Marking the coordinator 2147483647 dead" 将始终放入我们的日志中。
这基本上是你无法到达Kafka。
在我的例子中，我是 运行 Kafka in vagrant box，如果我启动 VPN，它会刷新
vagrant ip，因此它无法连接到它。
可能的解决方案： 在这种情况下，请停止 VPN 并启动您的 vagrant。
我今天遇到这个问题并解决了它（暂时，我可以补充一下）。我已经发布了关于我是如何做到的答案 here。
我遇到过同样的问题。最后在遵循 Shannon 关于 TRACING 日志的建议之后，我使用了：
logging.level.org.apache.kafka=TRACE
发现我的客户试图将 Euler:9092 解析为协调器...本地名称！！
所以我注释掉并更改了 server.properties 文件中的 listeners 和 advertised.listeners 值。
它现在正在工作！ :-)
这也可能与较长的垃圾收集停止世界阶段有关。在我的例子中，我在 > 10 秒的 GC 之后遇到了这条消息。
这个错误多发生在coordinator和consumer之间有冲突的时候。你应该做的第一件事是在 server.properties 中暴露监听端口，其次你需要删除 kafka-logs 下的所有日志。不要忘记在这些步骤之后重新启动服务器和 zookeeper。它将解决问题。