docker-compose kafka - 本地计算机客户端无法向 kafka 生成消息

Question

我已经阅读了很多类似的主题，但他们无法在这里回答我的问题。

尝试运行一些简短的集成测试，我正在使用 docker-compose 3，一个单节点 kafka。在客户端，我使用 Go shopify/sarama 来消费/生产

zookeeper:
  image: confluentinc/cp-zookeeper:5.2.2
  hostname: zookeeper
  container_name: zookeeper
  ports:
    - "2181:2181"
  environment:
    ZOOKEEPER_CLIENT_PORT: 2181
    ZOOKEEPER_TICK_TIME: 2000
kafka:
  image: confluentinc/cp-enterprise-kafka:5.2.2
  hostname: kafka
  container_name: kafka
  depends_on:
    - zookeeper
  ports:
    - "29092:29092"
  expose:
    - 9092
  environment:
    KAFKA_BROKER_ID: 1
    KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
    KAFKA_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
    KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
    KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
    KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

我有另一个来自 docker-compose 的容器可以收听

- "BROKERS_URL=kafka:9092"

消费者工作正常：

Sarama consumer up and running. {"brokers": ["kafka:9092"], "topics": ["validated"], "group": "event-service"}

但在生产者方面，运行直接从我的机器上宁：

kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

producer, err := sarama.NewSyncProducer([]string{"http://localhost:29092"}, nil)
...
msg := &sarama.ProducerMessage{
    Topic: "validated",
    Key:   sarama.StringEncoder(""),
    Value: sarama.ByteEncoder(payload),
}

partition, offset, err := producer.SendMessage(msg)
...

这里没有什么奇怪/夸张的东西，但它不起作用，我很困惑。

还有： nc -vz localhost 29092

Connection to localhost port 29092 [tcp/*] succeeded!

Answer 1

而不是

    KAFKA_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092

你需要

    KAFKA_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://0.0.0.0:29092

使用 kafkacat 从我的主机测试连接表明这有效：

➜ kafkacat -b localhost:29092 -L
Metadata for all topics (from broker 1: localhost:29092/1):
 1 brokers:
  broker 1 at localhost:29092 (controller)
 0 topics:

此区别在于侦听器绑定到所有可用接口 (0.0.0.0)。使用您的原始配置，它绑定到 localhost 的环回接口 (lo)，因此只接受此流量而不接受外部流量。

docker-compose kafka - 本地计算机客户端无法向 kafka 生成消息

docker-compose kafka - local machine client cannot produce message to kafka

apache-kafka

docker

sarama