如何通过 Flume 将键值发送到 kafka
how to send key value to kafka via Flume
我可以通过Flume向kafka发送消息，但是如何向kafka发送键值消息？
比如我使用tailf日志文件作为源，如何发送键值消息？
# Describe/configure the source
a1.sources.r1.type = exec
a1.sources.r1.command =tail -F /apps/logs/mysql_status.log
你必须使用 Flume headers
Kafka Sink uses the topic and key properties from the FlumeEvent headers to send events to Kafka. If topic exists in the headers, the event will be sent to that specific topic, overriding the topic configured for the Sink. If key exists in the headers, the key will used by Kafka to partition the data between the topic partitions
Headers可以加上拦截器
.http://flume.apache.org/releases/content/1.9.0/FlumeUserGuide.html#flume-interceptors
我可以通过Flume向kafka发送消息，但是如何向kafka发送键值消息？
比如我使用tailf日志文件作为源，如何发送键值消息？
# Describe/configure the source
a1.sources.r1.type = exec
a1.sources.r1.command =tail -F /apps/logs/mysql_status.log
你必须使用 Flume headers
Kafka Sink uses the topic and key properties from the FlumeEvent headers to send events to Kafka. If topic exists in the headers, the event will be sent to that specific topic, overriding the topic configured for the Sink. If key exists in the headers, the key will used by Kafka to partition the data between the topic partitions
Headers可以加上拦截器
.http://flume.apache.org/releases/content/1.9.0/FlumeUserGuide.html#flume-interceptors