如果 ISR 小于复制因子并且生产者 acks 设置为 all,生产者将等待多少个 acks?
If ISR is less than replication factor and producer acks are set to all, How many acks will the producer wait for?
- RF = 3,ISR = 3,acks = 全部 >>> 发送成功
- RF = 3,ISR = 2,acks = 全部 >>> 发送成功
- RF = 3,ISR = 1,acks = 全部 >>> 发送成功
- RF = 3,ISR = 1,acks = all,min.isr.replicas = 3 >>> 发送成功!
那么,如果复制因子为4,ISR为3,producer acks设置为all,producer会等待多少acks?我尝试了不同的场景,真实的行为应该是什么?
当涉及到acks
设置时,话题本身的replication factor
只是起到了隐含的作用。正如在 Broker 配置 documentation 中所写并在下面引用的那样,min.insync.replicas
定义了在被视为成功写入之前在代理范围内成功复制的最小数量。
min.insync.replicas: When a producer sets acks to "all" (or "-1"), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful. If this minimum cannot be met, then the producer will raise an exception (either NotEnoughReplicas or NotEnoughReplicasAfterAppend).
When used together, min.insync.replicas and acks allow you to enforce greater durability guarantees. A typical scenario would be to create a topic with a replication factor of 3, set min.insync.replicas to 2, and produce with acks of "all". This will ensure that the producer raises an exception if a majority of replicas do not receive a write.
Type: int
Default: 1
Valid Values: [1,...]
Importance: high
Update Mode: cluster-wide
回答你的具体问题:如果replication factor
是4,min.insync.replicas
是3,produceracks
设置为all,那么producer会在在被视为成功写入之前集群。
如果您设置 acks=all
作为分区领导者的代理将等待所有同步副本复制数据。同步副本是 不远 分区领导者的副本。
不远的意思是:
在 Kafka 中,当消息被发送到主题分区时(首先接收到消息并将其存储在领导者中)并且如果该主题的复制因子大于 1,则副本代理向领导代理发送获取请求,并且该数据被复制到其他经纪人。如果 replica.lag.time.max.ms
从上次追上传递过来,则副本被视为不同步并从 ISR 列表中删除。 (它仍然是一个副本并获取消息,但领导代理不会等到它赶上并再次成为一个同步副本)
来自 Kafka 文档:
Configuration parameter replica.lag.time.max.ms now refers not just to
the time passed since last fetch request from replica, but also to
time since the replica last caught up. Replicas that are still
fetching messages from leaders but did not catch up to the latest
messages in replica.lag.time.max.ms will be considered out of sync.
代理配置中还有 min.insync.replicas
参数。它指定在 acks=all
.
时继续发送消息的最小同步副本数
min.insync.replicas: When a producer sets acks to "all" (or "-1"), min.insync.replicas specifies the minimum number of replicas that must
acknowledge a write for the write to be considered successful. If this
minimum cannot be met, then the producer will raise an exception
(either NotEnoughReplicas or NotEnoughReplicasAfterAppend). When used
together, min.insync.replicas and acks allow you to enforce greater
durability guarantees. A typical scenario would be to create a topic
with a replication factor of 3, set min.insync.replicas to 2, and
produce with acks of "all". This will ensure that the producer raises
an exception if a majority of replicas do not receive a write.
If replication factor is 4, ISR is 3 and producer acks are set to
all, How many acks will the producer wait for?
答案:作为主题分区领导者的 Broker 将等待 ISR 列表中的其他 3 个 Broker 复制数据并发送确认。如果 ISR 列表中的副本数少于 min.insync.replicas
,那么您的生产者将收到异常并且无法生成数据。
注意:您可以使用以下命令查看当前副本和 ISR 列表。
bin/kafka-topics.sh --bootstrap-server localhost:9092 --topic myTopic --describe
- RF = 3,ISR = 3,acks = 全部 >>> 发送成功
- RF = 3,ISR = 2,acks = 全部 >>> 发送成功
- RF = 3,ISR = 1,acks = 全部 >>> 发送成功
- RF = 3,ISR = 1,acks = all,min.isr.replicas = 3 >>> 发送成功!
那么,如果复制因子为4,ISR为3,producer acks设置为all,producer会等待多少acks?我尝试了不同的场景,真实的行为应该是什么?
当涉及到acks
设置时,话题本身的replication factor
只是起到了隐含的作用。正如在 Broker 配置 documentation 中所写并在下面引用的那样,min.insync.replicas
定义了在被视为成功写入之前在代理范围内成功复制的最小数量。
min.insync.replicas: When a producer sets acks to "all" (or "-1"), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful. If this minimum cannot be met, then the producer will raise an exception (either NotEnoughReplicas or NotEnoughReplicasAfterAppend). When used together, min.insync.replicas and acks allow you to enforce greater durability guarantees. A typical scenario would be to create a topic with a replication factor of 3, set min.insync.replicas to 2, and produce with acks of "all". This will ensure that the producer raises an exception if a majority of replicas do not receive a write.
Type: int
Default: 1
Valid Values: [1,...]
Importance: high
Update Mode: cluster-wide
回答你的具体问题:如果replication factor
是4,min.insync.replicas
是3,produceracks
设置为all,那么producer会在在被视为成功写入之前集群。
如果您设置 acks=all
作为分区领导者的代理将等待所有同步副本复制数据。同步副本是 不远 分区领导者的副本。
不远的意思是:
在 Kafka 中,当消息被发送到主题分区时(首先接收到消息并将其存储在领导者中)并且如果该主题的复制因子大于 1,则副本代理向领导代理发送获取请求,并且该数据被复制到其他经纪人。如果 replica.lag.time.max.ms
从上次追上传递过来,则副本被视为不同步并从 ISR 列表中删除。 (它仍然是一个副本并获取消息,但领导代理不会等到它赶上并再次成为一个同步副本)
来自 Kafka 文档:
Configuration parameter replica.lag.time.max.ms now refers not just to the time passed since last fetch request from replica, but also to time since the replica last caught up. Replicas that are still fetching messages from leaders but did not catch up to the latest messages in replica.lag.time.max.ms will be considered out of sync.
代理配置中还有 min.insync.replicas
参数。它指定在 acks=all
.
min.insync.replicas: When a producer sets acks to "all" (or "-1"), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful. If this minimum cannot be met, then the producer will raise an exception (either NotEnoughReplicas or NotEnoughReplicasAfterAppend). When used together, min.insync.replicas and acks allow you to enforce greater durability guarantees. A typical scenario would be to create a topic with a replication factor of 3, set min.insync.replicas to 2, and produce with acks of "all". This will ensure that the producer raises an exception if a majority of replicas do not receive a write.
If replication factor is 4, ISR is 3 and producer acks are set to all, How many acks will the producer wait for?
答案:作为主题分区领导者的 Broker 将等待 ISR 列表中的其他 3 个 Broker 复制数据并发送确认。如果 ISR 列表中的副本数少于 min.insync.replicas
,那么您的生产者将收到异常并且无法生成数据。
注意:您可以使用以下命令查看当前副本和 ISR 列表。
bin/kafka-topics.sh --bootstrap-server localhost:9092 --topic myTopic --describe