来自 Kafka 消息的日志偏移量、分区和主题

Question

我想从 Kafka 的消息中记录或打印偏移量、分区和主题。 我可以打印消息值，但我想使用 python 查看来自 Kafka 的偏移量和分区，以便我可以调试我的代码

from kafka import KafkaConsumer

consumer = KafkaConsumer('my-topic',
                         group_id='my-group',
                         bootstrap_servers=['localhost:9092'])
for message in consumer:
    print(f"Message is {message.value()}") 

Answer 1

该数据应该可以使用点表示法

The consumer iterator returns ConsumerRecords, which are simple namedtuples that expose basic message attributes: topic, partition, offset, key, and value:

https://github.com/dpkp/kafka-python

Answer 2

from kafka import KafkaConsumer

consumer = KafkaConsumer('my-topic',
                         group_id='my-group',
                         bootstrap_servers=['localhost:9092'])
for message in consumer:
    logger.debug(f"Consumer data: offset-{message.offset()} partition-{message.partition()} topic-{message.topic()}")
    print(f"Message is {message.value()}")