Kafka s3 源连接器：无法实例化分区程序 class

Question

当我尝试创建 s3 源连接器时出现错误 无法实例化分区程序 class。做了什么：安装 confluent-hub 和 confluentinc/kafka-connect-s3-source，导出了 CLASSHPATH。 (1.0.1 为最新版本)

$ confluent-hub install --no-prompt confluentinc/kafka-connect-s3-source:1.0.1
$ export CLASSPATH=/connector/share/confluent-hub-components/confluentinc-kafka-connect-s3-source/lib/*

连接器设置是文档中的默认设置 (connector.properties)

name=s3-source
tasks.max=1
connector.class=io.confluent.connect.s3.source.S3SourceConnector
s3.bucket.name=confluent-kafka-connect-s3-testing
format.class=io.confluent.connect.s3.format.avro.AvroFormat
confluent.license=
confluent.topic.bootstrap.servers=kafka:9092
confluent.topic.replication.factor=1

transforms=AddPrefix
transforms.AddPrefix.type=org.apache.kafka.connect.transforms.RegexRouter
transforms.AddPrefix.regex=.*
transforms.AddPrefix.replacement=copy_of_[=12=]

详细错误

$ connect-standalone.sh worker.properties connector.properties
[2019-10-16 12:36:02,410] INFO Kafka version: 2.3.0 (org.apache.kafka.common.utils.AppInfoParser:117)
[2019-10-16 12:36:02,411] INFO Kafka commitId: fc1aaa116b661c8a (org.apache.kafka.common.utils.AppInfoParser:118)
[2019-10-16 12:36:02,412] INFO Kafka startTimeMs: 1571229362410 (org.apache.kafka.common.utils.AppInfoParser:119)
[2019-10-16 12:36:02,675] INFO License for single cluster, single node (io.confluent.license.LicenseManager:417)
[2019-10-16 12:36:02,683] INFO Closing License Store (io.confluent.license.LicenseStore:197)
[2019-10-16 12:36:02,683] INFO Stopping KafkaBasedLog for topic _confluent-command (org.apache.kafka.connect.util.KafkaBasedLog:164)
[2019-10-16 12:36:02,686] INFO [Producer clientId=s3-source-license-manager] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms. (org.apache.kafka.clients.producer.KafkaProducer:1153)
[2019-10-16 12:36:02,701] INFO Stopped KafkaBasedLog for topic _confluent-command (org.apache.kafka.connect.util.KafkaBasedLog:190)
[2019-10-16 12:36:02,702] INFO Closed License Store (io.confluent.license.LicenseStore:199)
[2019-10-16 12:36:02,704] ERROR WorkerConnector{id=s3-source} Error while starting connector (org.apache.kafka.connect.runtime.WorkerConnector:119)
org.apache.kafka.connect.errors.ConnectException: Failed to instantiate Partitioner class 
    at io.confluent.connect.s3.source.S3SourceConnectorConfig.getPartitioner(S3SourceConnectorConfig.java:612)
    at io.confluent.connect.s3.source.S3SourceConnector.doStart(S3SourceConnector.java:94)
    at io.confluent.connect.s3.source.S3SourceConnector.start(S3SourceConnector.java:86)
    at org.apache.kafka.connect.runtime.WorkerConnector.doStart(WorkerConnector.java:111)
    at org.apache.kafka.connect.runtime.WorkerConnector.start(WorkerConnector.java:136)
    at org.apache.kafka.connect.runtime.WorkerConnector.transitionTo(WorkerConnector.java:196)
    at org.apache.kafka.connect.runtime.Worker.startConnector(Worker.java:252)
    at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.startConnector(StandaloneHerder.java:293)
    at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:209)
    at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:115)

我不熟悉 Java 但现在我想查看 jar 中的源代码。任何帮助都会有所帮助，并提前致谢。

Answer 1

听起来你的安装有点麻烦。要在 Docker 下运行 Kafka Connect，您应该使用专用图像，例如 confluentinc/cp-kafka-connect。

要查看使用 Docker 部署的 Kafka Connect 示例，请查看 http://rmoff.dev/bbuzz19_demo-code and accompanying talk and slides

Kafka s3 源连接器：无法实例化分区程序 class

Kafka s3 source connector: Failed to instantiate Partitioner class

apache-kafka-connect

confluent-platform