Kafka 有持久订阅功能吗?

Does Kafka have Durable Subscriptions feature?

我有兴趣在我的一个项目中使用 Kafka,但要求消息传递代理必须在其中一个订阅者(消费者)断开连接时保留消息。

我看到 JMS 有这个 feature

网站上说 Kafka 有 durability 特性。

和JMS是一样的还是有不同的意思?

消费者从卡夫卡(经纪人)拉取数据。消费者指定它想要收集数据的位置的偏移量。如果消费者断开连接并返回,它可以从它离开的地方继续。它还可以从较早的点开始使用数据(更改偏移量)。

Kafka 确实支持持久消费者风格模式,但有几种方法可以实现它。

首先您需要了解偏移量和消费者位置

的概念

Kafka maintains a numerical offset for each record in a partition. This offset acts as a unique identifier of a record within that partition, and also denotes the position of the consumer in the partition. For example, a consumer which is at position 5 has consumed records with offsets 0 through 4 and will next receive the record with offset 5. There are actually two notions of position relevant to the user of the consumer: The position of the consumer gives the offset of the next record that will be given out. It will be one larger than the highest offset the consumer has seen in that partition. It automatically advances every time the consumer receives messages in a call to poll(Duration).

The committed position is the last offset that has been stored securely. Should the process fail and restart, this is the offset that the consumer will recover to. The consumer can either automatically commit offsets periodically; or it can choose to control this committed position manually by calling one of the commit APIs (e.g. commitSync and commitAsync).

Kafka服务器端或客户端的偏移量可以是stored/persisted:

  1. Kafka Server persists/holds消费者位置,本例中有2个子选项:

    • 消费者显式提交消息消费

    • Consumer自动提交消息消费

  2. 客户端应用程序persists/holds 消费者立场

这都是根据 https://kafka.apache.org/22/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html