定义 Camel S3 源连接器时出错

Question

我正在尝试在我们的融合环境中定义一个 Camel S3 源连接器。

这是我正在使用的配置

{
    "name": "CamelAWSS3SourceConnector",
    "config": {
        "connector.class": "org.apache.camel.kafkaconnector.awss3.CamelAwss3SourceConnector",
        "key.converter": "org.apache.kafka.connect.storage.StringConverter",
        "value.converter": "org.apache.camel.kafkaconnector.awss3.converters.S3ObjectConverter",
        "camel.source.maxPollDuration": "10000",
        "topics": "TEST-S3-SOURCE-POC",
        "camel.source.path.bucketNameOrArn": "json-poc",
        "camel.component.aws-s3.region": "us-east-1",
        "tasks.max": "2",
        "camel.source.endpoint.autocloseBody": "true"
    }
}

这是我在尝试定义连接器时收到的错误

{
    "error_code": 405,
    "message": "HTTP 405 Method Not Allowed"
} {
    "name": "CamelAWSS3SourceConnector",
    "connector": {
        "state": "RUNNING",
        "worker_id": "confluent-connect-server2:8083"
    },
    "tasks": [{
        "id": 0,
        "state": "FAILED",
        "worker_id": "confluent-connect-server2",
        "trace": "org.apache.kafka.connect.errors.ConnectException: Failed to create and start Camel context
    at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:118)
    at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:215)
    at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:184)
    at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:234)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Option bucketNameOrArn is required when creating endpoint uri with syntax aws-s3://bucketNameOrArn
    at org.apache.camel.support.component.EndpointUriFactorySupport.buildPathParameter(EndpointUriFactorySupport.java:53)
    at org.apache.camel.component.aws.s3.S3EndpointUriFactory.buildUri(S3EndpointUriFactory.java:103)
    at org.apache.camel.kafkaconnector.utils.TaskHelper.buildUrl(TaskHelper.java:68)
    at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:98)
    ... 8 more"
    }],
    "type": "source"
}

出现上述错误的原因是什么？

有人告诉我，由于连接服务器是一个 ec2 实例，因此我不必在此处定义 AWS 参数。对吗？

谢谢

注意： 想补充一点，有两个连接服务器，输出中只显示其中一个的错误

Answer 1

只是想post在这里回答，以防其他人遇到这个问题。

问题是使用 PUT 添加或更新连接器时，curl 请求的 url 格式需要 http(s)://<serverurl>/<connectorName>/config。我的 json 包含键“name:”，而 url 只是 http(s)://。最初，我使用 POST 创建了连接器，然后尝试使用 PUT 更新（实际上是添加）实际上没有更新的 bucketNameOrArn 键。

定义 Camel S3 源连接器时出错

Error when defining Camel S3 source connector

apache-kafka

apache-kafka-connect

apache-camel-aws

s3-kafka-connector