Kafka - 集群范围是什么意思
Kafka - What does it mean by cluster-wide
我设置了 3 个代理 kafka。我正在更新“min.insync.replicas”属性，我查看了文档，
min.insync.replicas
Update Mode: cluster-wide
- “集群范围”在此处指定什么 - 如果我在一个代理中更新 属性 是否可以，或者是否需要在所有代理设置文件中更新 (server.properties)
- 如何使用 kafka-config -
更新“集群范围”属性
All configs that are configurable at cluster level may also be
configured at per-broker level (e.g. for testing). If a config value
is defined at different levels, the following order of precedence is
used: Dynamic per-broker config stored in ZooKeeper Dynamic
cluster-wide default config stored in ZooKeeper Static broker config
from server.properties Kafka default, see broker configs
From Kafka version 1.1 onwards, some of the broker configs can be
updated without restarting the broker. See the Dynamic Update Mode
column in Broker Configs for the update mode of each broker config.
read-only: Requires a broker restart for update per-broker: May be
updated dynamically for each broker cluster-wide: May be updated
dynamically as a cluster-wide default. May also be updated as a
per-broker value for testing.
min.insync.replicas表示没有。需要同步的主题分区的副本数。
What does "cluster-wide" specify here - Is it ok if i update the
property in one broker or is it required to be updated in all broker
settings file ( server.properties)
集群范围的配置应该适用于所有代理。所以更新一次应该反映其他代理，否则 cluster-wide 配置和 per-broker 配置之间可能没有区别。
配置 min.insync.replicas 在主题级别和集群级别均可配置。
当我们在主题级别设置它时，它只适用于该主题，无论主题分区在哪个代理中。
当我们在代理级别设置它时，它将是所有主题的默认设置，除非另有说明。
How to update "cluster-wide" property using kafka-config?
如果您想设置主题，此处的配置实体类型将为topics。
如果要设置在broker级别，那就是brokers
这是完整列表（截至 2.3.1）
--add-config <String> Key Value pairs of configs to add.
Square brackets can be used to group
values which contain commas: 'k1=v1,
k2=[v1,v2,v2],k3=v3'. The following
is a list of valid configurations:
For entity-type 'topics':
cleanup.policy
compression.type
confluent.tier.enable
confluent.tier.local.hotset.bytes
confluent.tier.local.hotset.ms
delete.retention.ms
file.delete.delay.ms
flush.messages
flush.ms
follower.replication.throttled.
replicas
index.interval.bytes
leader.replication.throttled.replicas
max.compaction.lag.ms
max.message.bytes
message.downconversion.enable
message.format.version
message.timestamp.difference.max.ms
message.timestamp.type
min.cleanable.dirty.ratio
min.compaction.lag.ms
min.insync.replicas
preallocate
retention.bytes
retention.ms
segment.bytes
segment.index.bytes
segment.jitter.ms
segment.ms
unclean.leader.election.enable
For entity-type 'brokers':
log.message.timestamp.type
ssl.client.auth
log.retention.ms
sasl.login.refresh.window.jitter
sasl.kerberos.ticket.renew.window.
factor
log.preallocate
log.index.size.max.bytes
sasl.login.refresh.window.factor
ssl.truststore.type
ssl.keymanager.algorithm
log.cleaner.io.buffer.load.factor
sasl.login.refresh.min.period.seconds
ssl.key.password
background.threads
log.retention.bytes
ssl.trustmanager.algorithm
log.segment.bytes
max.connections.per.ip.overrides
log.cleaner.delete.retention.ms
log.segment.delete.delay.ms
min.insync.replicas
ssl.keystore.location
ssl.cipher.suites
log.roll.jitter.ms
log.cleaner.backoff.ms
sasl.jaas.config
principal.builder.class
log.flush.interval.ms
confluent.tier.enable
log.cleaner.max.compaction.lag.ms
max.connections
log.cleaner.dedupe.buffer.size
log.flush.interval.messages
advertised.listeners
num.io.threads
listener.security.protocol.map
log.message.downconversion.enable
sasl.enabled.mechanisms
sasl.login.refresh.buffer.seconds
ssl.truststore.password
listeners
metric.reporters
ssl.protocol
sasl.kerberos.ticket.renew.jitter
ssl.keystore.password
sasl.mechanism.inter.broker.protocol
log.cleanup.policy
sasl.kerberos.principal.to.local.rules
sasl.kerberos.min.time.before.relogin
num.recovery.threads.per.data.dir
log.cleaner.io.max.bytes.per.second
log.roll.ms
confluent.tier.local.hotset.ms
ssl.endpoint.identification.algorithm
unclean.leader.election.enable
message.max.bytes
log.cleaner.threads
log.cleaner.io.buffer.size
max.connections.per.ip
sasl.kerberos.service.name
ssl.provider
follower.replication.throttled.rate
log.index.interval.bytes
log.cleaner.min.compaction.lag.ms
log.message.timestamp.difference.max.
ms
ssl.enabled.protocols
confluent.tier.local.hotset.bytes
log.cleaner.min.cleanable.ratio
replica.alter.log.dirs.io.max.bytes.
per.second
ssl.keystore.type
ssl.secure.random.implementation
ssl.truststore.location
sasl.kerberos.kinit.cmd
leader.replication.throttled.rate
num.network.threads
compression.type
num.replica.fetchers
For entity-type 'users':
request_percentage
producer_byte_rate
SCRAM-SHA-256
SCRAM-SHA-512
consumer_byte_rate
For entity-type 'clients':
request_percentage
producer_byte_rate
consumer_byte_rate
Entity types 'users' and 'clients' may
be specified together to update
config for clients of a specific
user.
存储在 ZooKeeper (Reference) 中的动态集群范围默认配置，因此适用于所有代理。
我设置了 3 个代理 kafka。我正在更新“min.insync.replicas”属性，我查看了文档，
min.insync.replicas
Update Mode: cluster-wide
- “集群范围”在此处指定什么 - 如果我在一个代理中更新 属性 是否可以，或者是否需要在所有代理设置文件中更新 (server.properties)
- 如何使用 kafka-config - 更新“集群范围”属性
All configs that are configurable at cluster level may also be configured at per-broker level (e.g. for testing). If a config value is defined at different levels, the following order of precedence is used: Dynamic per-broker config stored in ZooKeeper Dynamic cluster-wide default config stored in ZooKeeper Static broker config from server.properties Kafka default, see broker configs
From Kafka version 1.1 onwards, some of the broker configs can be updated without restarting the broker. See the Dynamic Update Mode column in Broker Configs for the update mode of each broker config. read-only: Requires a broker restart for update per-broker: May be updated dynamically for each broker cluster-wide: May be updated dynamically as a cluster-wide default. May also be updated as a per-broker value for testing.
min.insync.replicas表示没有。需要同步的主题分区的副本数。
What does "cluster-wide" specify here - Is it ok if i update the property in one broker or is it required to be updated in all broker settings file ( server.properties)
集群范围的配置应该适用于所有代理。所以更新一次应该反映其他代理，否则 cluster-wide 配置和 per-broker 配置之间可能没有区别。
配置 min.insync.replicas 在主题级别和集群级别均可配置。
当我们在主题级别设置它时，它只适用于该主题，无论主题分区在哪个代理中。
当我们在代理级别设置它时，它将是所有主题的默认设置，除非另有说明。
How to update "cluster-wide" property using kafka-config?
如果您想设置主题，此处的配置实体类型将为topics。
如果要设置在broker级别，那就是brokers
这是完整列表（截至 2.3.1）
--add-config <String> Key Value pairs of configs to add.
Square brackets can be used to group
values which contain commas: 'k1=v1,
k2=[v1,v2,v2],k3=v3'. The following
is a list of valid configurations:
For entity-type 'topics':
cleanup.policy
compression.type
confluent.tier.enable
confluent.tier.local.hotset.bytes
confluent.tier.local.hotset.ms
delete.retention.ms
file.delete.delay.ms
flush.messages
flush.ms
follower.replication.throttled.
replicas
index.interval.bytes
leader.replication.throttled.replicas
max.compaction.lag.ms
max.message.bytes
message.downconversion.enable
message.format.version
message.timestamp.difference.max.ms
message.timestamp.type
min.cleanable.dirty.ratio
min.compaction.lag.ms
min.insync.replicas
preallocate
retention.bytes
retention.ms
segment.bytes
segment.index.bytes
segment.jitter.ms
segment.ms
unclean.leader.election.enable
For entity-type 'brokers':
log.message.timestamp.type
ssl.client.auth
log.retention.ms
sasl.login.refresh.window.jitter
sasl.kerberos.ticket.renew.window.
factor
log.preallocate
log.index.size.max.bytes
sasl.login.refresh.window.factor
ssl.truststore.type
ssl.keymanager.algorithm
log.cleaner.io.buffer.load.factor
sasl.login.refresh.min.period.seconds
ssl.key.password
background.threads
log.retention.bytes
ssl.trustmanager.algorithm
log.segment.bytes
max.connections.per.ip.overrides
log.cleaner.delete.retention.ms
log.segment.delete.delay.ms
min.insync.replicas
ssl.keystore.location
ssl.cipher.suites
log.roll.jitter.ms
log.cleaner.backoff.ms
sasl.jaas.config
principal.builder.class
log.flush.interval.ms
confluent.tier.enable
log.cleaner.max.compaction.lag.ms
max.connections
log.cleaner.dedupe.buffer.size
log.flush.interval.messages
advertised.listeners
num.io.threads
listener.security.protocol.map
log.message.downconversion.enable
sasl.enabled.mechanisms
sasl.login.refresh.buffer.seconds
ssl.truststore.password
listeners
metric.reporters
ssl.protocol
sasl.kerberos.ticket.renew.jitter
ssl.keystore.password
sasl.mechanism.inter.broker.protocol
log.cleanup.policy
sasl.kerberos.principal.to.local.rules
sasl.kerberos.min.time.before.relogin
num.recovery.threads.per.data.dir
log.cleaner.io.max.bytes.per.second
log.roll.ms
confluent.tier.local.hotset.ms
ssl.endpoint.identification.algorithm
unclean.leader.election.enable
message.max.bytes
log.cleaner.threads
log.cleaner.io.buffer.size
max.connections.per.ip
sasl.kerberos.service.name
ssl.provider
follower.replication.throttled.rate
log.index.interval.bytes
log.cleaner.min.compaction.lag.ms
log.message.timestamp.difference.max.
ms
ssl.enabled.protocols
confluent.tier.local.hotset.bytes
log.cleaner.min.cleanable.ratio
replica.alter.log.dirs.io.max.bytes.
per.second
ssl.keystore.type
ssl.secure.random.implementation
ssl.truststore.location
sasl.kerberos.kinit.cmd
leader.replication.throttled.rate
num.network.threads
compression.type
num.replica.fetchers
For entity-type 'users':
request_percentage
producer_byte_rate
SCRAM-SHA-256
SCRAM-SHA-512
consumer_byte_rate
For entity-type 'clients':
request_percentage
producer_byte_rate
consumer_byte_rate
Entity types 'users' and 'clients' may
be specified together to update
config for clients of a specific
user.
存储在 ZooKeeper (Reference) 中的动态集群范围默认配置，因此适用于所有代理。