使用 docker-compose 创建 Kafka 主题

Question

我正在开始我的第一个 docker 项目，尝试设置 zookeeper 和 Kafka。

我有以下 docker-compose.yml:

version: "3.8"

networks:
    mynet:

services:

  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"
   networks:
      - mynet
      
  kafka:
    image: wurstmeister/kafka:2.12-2.4.0
    ports:
      - "9092:9092"
    expose:
      - "9093"
    environment:
      KAFKA_LISTENERS: INSIDE://0.0.0.0:9093,OUTSIDE://0.0.0.0:9092
      KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9093,OUTSIDE://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      # Create a topic NAME:PARTITION:REPLICAS
      KAFKA_CREATE_TOPICS: "example-topic:1:1"
   networks:
      - mynet
      
  kafka-manager:
    image: sheepkiller/kafka-manager:latest
    environment:
      ZK_HOSTS: "zookeeper:2181"
    ports:
      - 9000:9000
    networks:
      - mynet

我执行docker-compose.yml： sudo docker-compose up -d

为了检查我是否创建了主题，我通过以下方式访问 Kafka shell： sudo docker exec -it <<CONTAINER ID>> sh

在这个 shell 我去： cd opt/kafka

并执行： bin/kafka-topics.sh --list --zookeeper localhost:2181

并且输出是一个超时错误：

Exception in thread "main" kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
    at kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected(ZooKeeperClient.scala:259)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:253)
    at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:255)
    at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:113)
    at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1858)
    at kafka.admin.TopicCommand$ZookeeperTopicService$.apply(TopicCommand.scala:321)
    at kafka.admin.TopicCommand$.main(TopicCommand.scala:54)
    at kafka.admin.TopicCommand.main(TopicCommand.scala)

Answer 1

Zookeeper 运行不在 Kafka 容器中。

你应该使用 --bootstrap-server localhost:9092 无论如何，因为 Zookeeper 参数已被弃用

使用 docker-compose 创建 Kafka 主题

Creating Kafka Topic with docker-compose

apache-kafka

docker

docker-compose

apache-zookeeper