Java 中的 Kafka Connect 动态连接器

Kafka Connect Dynamic connectors in Java

这就是开发人员指南对动态连接器的看法https://docs.confluent.io/current/connect/devguide.html#dynamic-connectors

Not all connectors have a static set of partitions, so Connector implementations are also responsible for monitoring the external system for any changes that might require reconfiguration. For example, in the JDBCSourceConnector example, the Connector might assign a set of tables to each Task. When a new table is created, it must discover this so it can assign the new table to one of the Tasks by updating its configuration. When it notices a change that requires reconfiguration (or a change in the number of Tasks), it notifies the framework and the framework updates any corresponding Tasks

我不确定我应该怎么做。

连接器 class(扩展 org.apache.kafka.connect.source.SourceConnector)实现了 taskConfigs(int)start(Map<String,String>)stop()config()version()taskClass()。它没有 poll() 或类似的方法。

我是否应该在 start(Map<String,String>) 中生成一个线程来监视外部系统?

如果检测到变化,我该如何 'notify the framework'。 Java SDK 中是否有一些 API 调用执行此操作,或者这是否意味着我必须先调用 stop() 然后调用 start()

谢谢。

Am I suppose to spawn a thread in start(Map<String,String>) that monitors the external system?

是的,正是。

how do I 'notify the framework'.

通过 SourceConnector 上下文的 requestTaskReconfiguration() 方法。