卡夫卡满怀期待地关闭。无法创建许可证主题

Question

在 Kafka 启动时，多条消息被记录到 kafka/logs/kafkaServer.out 并包含：

INFO [Admin Manager on Broker 0]: Error processing create topic request CreatableTopic(name='_confluent-license', numPartitions=1, replicationFactor=3, assignments=[], configs=[CreateableTopicConfig(name='cleanup.policy', value='compact'), CreateableTopicConfig(name='min.insync.replicas', value='2')]) (kafka.server.AdminManager)

大约 15 分钟后，Kafka 关闭并输出到 kafka/logs/kafkaServer.out 是：

org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 1.
[2020-12-08 04:04:15,951] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown
(kafka.server.KafkaServer) org.apache.kafka.common.errors.TimeoutException: License topic could not be created Caused by: org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 1. [2020-12-08 04:04:15,952] INFO [KafkaServer id=0] shutting down (kafka.server.KafkaServer)

似乎 Kafka 关闭是因为主题 _confluent-license 的复制因子设置为 3？我不是在创建主题 _confluent-license，它是作为 Kafka 启动的一部分创建的以进行许可检查吗？

为了修复，我修改了 /v5.5.0/etc/kafka/server.properties 以便内部主题的复制因子为 1：

############################# Internal Topic Settings  #############################
# The replication factor for the group metadata internal topics "__consumer_offsets" and "__transaction_state"
# For anything other than development testing, a value greater than 1 is recommended for to ensure availability such as 3.
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1

而不是 3 :

#offsets.topic.replication.factor=3
#transaction.state.log.replication.factor=3

但这并不能解决问题，并且会生成相同的日志。 __consumer_offsets 的复制因子仍然是 3。如何将主题 _confluent-license 的复制因子从 3 降低到 1？或者是否存在导致 Kafka 关闭的其他问题？

Answer 1

您应该将属性 confluent.license.topic.replication.factor 默认更改为 3。

Answer 2

(kafka.server.KafkaServer) org.apache.kafka.common.errors.TimeoutException: License topic could not be created Caused by: org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 1. [2020-12-08 04:04:15,952] INFO [KafkaServer id=0] shutting down (kafka.server.KafkaServer)

上述错误是由于 许可主题的默认复制因子为 3。如果您只有 1 个代理，则可以将 confluent.license.topic.replication.factor 配置为等于 1。相同的文档是 here.

[2020-12-08 07:46:02,241] ERROR Error checking or creating metrics topic (io.confluent.metrics.reporter.ConfluentMetricsReporter) org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 2 larger than available brokers: 1.

以上错误是由于 Confluent Metrics Reporter 被启用。指标主题的 复制因子默认为 3，如果您只有一个代理，可以将 confluent.metrics.reporter.topic.replicas 配置为 1。相同的文档是 here.

Answer 3

属性 confluent.license.topic.replication.factor 未按预期工作。

相反，我们可以尝试这些属性中的任何一个 confluent.license.replication.factor=1 或 confluent.license.admin.replication.factor=1 如果经纪人的数量是 1。这个属性不在文档中。

卡夫卡满怀期待地关闭。无法创建许可证主题

Kafka expectantly shutting down. License topic could not be created

apache-kafka

confluent-platform