无法 运行 Kafka 以分布式模式连接 - 尝试创建/查找主题时出错 'connect-offsets'
Not able to run Kafka Connect in distributed mode - Error while attempting to create/ find topics 'connect-offsets'
[2017-08-31 10:15:20,715] WARN The configuration
'internal.key.converter' was supplied but isn't a known config.
(org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31
10:15:20,715] WARN The configuration
'status.storage.replication.factor' was supplied but isn't a known
config. (org.apache.kafka.clients.admin.AdminClientConfig:231)
[2017-08-31 10:15:20,715] WARN The configuration
'internal.value.converter.schemas.enable' was supplied but isn't a
known config. (org.apache.kafka.clients.admin.AdminClientConfig:231)
[2017-08-31 10:15:20,715] WARN The configuration
'value.converter.schemas.enable' was supplied but isn't a known
config. (org.apache.kafka.clients.admin.AdminClientConfig:231)
[2017-08-31 10:15:20,715] WARN The configuration
'internal.value.converter' was supplied but isn't a known config.
(org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31
10:15:20,715] WARN The configuration
'offset.storage.replication.factor' was supplied but isn't a known
config. (org.apache.kafka.clients.admin.AdminClientConfig:231)
[2017-08-31 10:15:20,716] WARN The configuration
'offset.storage.topic' was supplied but isn't a known config.
(org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31
10:15:20,716] WARN The configuration 'value.converter' was supplied
but isn't a known config.
(org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31
10:15:20,716] WARN The configuration 'key.converter' was supplied but
isn't a known config.
(org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31
10:15:20,768] INFO jetty-9.2.15.v20160210
(org.eclipse.jetty.server.Server:327) Aug 31, 2017 10:15:21 AM
org.glassfish.jersey.internal.Errors logErrors WARNING: The following
warnings have been detected: WARNING: The (sub)resource method
listConnectors in
org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource
contains empty path annotation. WARNING: The (sub)resource method
createConnector in
org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource
contains empty path annotation. WARNING: The (sub)resource method
listConnectorPlugins in
org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource
contains empty path annotation. WARNING: The (sub)resource method
serverInfo in
org.apache.kafka.connect.runtime.rest.resources.RootResource contains
empty path annotation.
[2017-08-31 10:15:21,142] INFO Started
o.e.j.s.ServletContextHandler@63f02de0{/,null,AVAILABLE}
(org.eclipse.jetty.server.handler.ContextHandler:744) [2017-08-31
10:15:21,149] INFO Started
ServerConnector@61b06738{HTTP/1.1}{0.0.0.0:8083}
(org.eclipse.jetty.server.ServerConnector:266) [2017-08-31
10:15:21,149] INFO Started @13584ms
(org.eclipse.jetty.server.Server:379) [2017-08-31 10:15:21,150] INFO
REST server listening at http://172.28.193.126:8083/, advertising URL
http://172.28.193.126:8083/
(org.apache.kafka.connect.runtime.rest.RestServer:150) [2017-08-31
10:15:21,150] INFO Kafka Connect started
(org.apache.kafka.connect.runtime.Connect:55) [2017-08-31
10:15:21,468] ERROR Uncaught exception in herder work thread, exiting:
(org.apache.kafka.connect.runtime.distributed.DistributedHerder:206)
org.apache.kafka.connect.errors.ConnectException: Error while
attempting to create/find topic(s) 'connect-offsets' at
org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:245)
at
org.apache.kafka.connect.storage.KafkaOffsetBackingStore.run(KafkaOffsetBackingStore.java:99)
at
org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:126)
at
org.apache.kafka.connect.storage.KafkaOffsetBackingStore.start(KafkaOffsetBackingStore.java:109)
at org.apache.kafka.connect.runtime.Worker.start(Worker.java:146) at
org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:99)
at
org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:194)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) Caused by:
java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.UnknownServerException: The server
experienced an unexpected error when processing the request at
org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at
org.apache.kafka.common.internals.KafkaFutureImpl.access[=12=]0(KafkaFutureImpl.java:32)
at
org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
at
org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:213)
at
org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:226)
Kafka Connect 0.11.0.0 无法在早于 0.10.1.0 的代理上创建主题，因为这些旧代理中使用的协议不支持主题管理。
因此，如果您有一个较旧的代理，您应该手动创建主题或依靠代理自动创建。不幸的是，Kafka Connect 0.11.0.0 中的 a bug 会阻止自动创建，因此在 0.11.0.1 可用之前，您必须手动创建主题。
在分布式模式下启动 Kafka Connect 时，它将尝试创建或期望多个主题出现在 Kafka 中。在我的情况下，自动主题创建已关闭，因此 Kafka Connect 无法在我 运行 所在的环境中创建主题。我自己创建这些主题解决了这个问题。您应该在您的属性文件（如下）中指定这些主题的名称。
offset.storage.topic
config.storage.topic
status.storage.topic
[2017-08-31 10:15:20,715] WARN The configuration 'internal.key.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,715] WARN The configuration 'status.storage.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,715] WARN The configuration 'internal.value.converter.schemas.enable' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,715] WARN The configuration 'value.converter.schemas.enable' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,715] WARN The configuration 'internal.value.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,715] WARN The configuration 'offset.storage.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,716] WARN The configuration 'offset.storage.topic' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,716] WARN The configuration 'value.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,716] WARN The configuration 'key.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,768] INFO jetty-9.2.15.v20160210 (org.eclipse.jetty.server.Server:327) Aug 31, 2017 10:15:21 AM org.glassfish.jersey.internal.Errors logErrors WARNING: The following warnings have been detected: WARNING: The (sub)resource method listConnectors in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation. WARNING: The (sub)resource method createConnector in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation. WARNING: The (sub)resource method listConnectorPlugins in org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource contains empty path annotation. WARNING: The (sub)resource method serverInfo in org.apache.kafka.connect.runtime.rest.resources.RootResource contains empty path annotation.
[2017-08-31 10:15:21,142] INFO Started o.e.j.s.ServletContextHandler@63f02de0{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:744) [2017-08-31 10:15:21,149] INFO Started ServerConnector@61b06738{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector:266) [2017-08-31 10:15:21,149] INFO Started @13584ms (org.eclipse.jetty.server.Server:379) [2017-08-31 10:15:21,150] INFO REST server listening at http://172.28.193.126:8083/, advertising URL http://172.28.193.126:8083/ (org.apache.kafka.connect.runtime.rest.RestServer:150) [2017-08-31 10:15:21,150] INFO Kafka Connect started (org.apache.kafka.connect.runtime.Connect:55) [2017-08-31 10:15:21,468] ERROR Uncaught exception in herder work thread, exiting: (org.apache.kafka.connect.runtime.distributed.DistributedHerder:206) org.apache.kafka.connect.errors.ConnectException: Error while attempting to create/find topic(s) 'connect-offsets' at org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:245) at org.apache.kafka.connect.storage.KafkaOffsetBackingStore.run(KafkaOffsetBackingStore.java:99) at org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:126) at org.apache.kafka.connect.storage.KafkaOffsetBackingStore.start(KafkaOffsetBackingStore.java:109) at org.apache.kafka.connect.runtime.Worker.start(Worker.java:146) at org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:99) at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:194) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.UnknownServerException: The server experienced an unexpected error when processing the request at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45) at org.apache.kafka.common.internals.KafkaFutureImpl.access[=12=]0(KafkaFutureImpl.java:32) at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89) at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:213) at org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:226)
Kafka Connect 0.11.0.0 无法在早于 0.10.1.0 的代理上创建主题，因为这些旧代理中使用的协议不支持主题管理。
因此，如果您有一个较旧的代理，您应该手动创建主题或依靠代理自动创建。不幸的是，Kafka Connect 0.11.0.0 中的 a bug 会阻止自动创建，因此在 0.11.0.1 可用之前，您必须手动创建主题。
在分布式模式下启动 Kafka Connect 时，它将尝试创建或期望多个主题出现在 Kafka 中。在我的情况下，自动主题创建已关闭，因此 Kafka Connect 无法在我 运行 所在的环境中创建主题。我自己创建这些主题解决了这个问题。您应该在您的属性文件（如下）中指定这些主题的名称。
offset.storage.topic
config.storage.topic
status.storage.topic