Quarkus 无法从内部连接到 kafka docker

Question

我创建了一个 quarkus 服务，它从一堆 Kstreams 中读取数据，加入它们，然后 post 将加入结果返回到 kafka 主题中。在开发过程中，我是运行来自 docker-compose 的 kafka 和 zookeeper，然后是运行我在开发模式下的 quarkus 服务：

mvn quarkus:dev

此时，一切正常。我可以毫无问题地连接到代理和 read/write Kstreams。

然后我尝试创建一个 docker 容器来运行此 quarkus 服务，但是当该服务在容器内运行时，它不会到达代理。我在 docker-compose 中尝试了几种不同的配置，但 none 有效。它只是无法连接到代理。

这是我的 Dockerfile：

####
# This Dockerfile is used in order to build a container that runs the Quarkus application in JVM mode
#
# Before building the container image run:
#
# mvn package
#
# Then, build the image with:
#
# docker build -f src/main/docker/Dockerfile.jvm -t connector .
#
# Then run the container using:
#
# docker run -i --rm -p 8080:8080 connector
#
# If you want to include the debug port into your docker image
# you will have to expose the debug port (default 5005) like this :  EXPOSE 8080 5050
#
# Then run the container using :
#
# docker run -i --rm -p 8080:8080 -p 5005:5005 -e JAVA_ENABLE_DEBUG="true" connector
#
###
FROM docker.internal/library/quarkus-base:latest

ARG RUN_JAVA_VERSION=1.3.8
ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en'

USER root
RUN apk update && apk add libstdc++

# Configure the JAVA_OPTIONS, you can add -XshowSettings:vm to also display the heap size.
ENV JAVA_OPTIONS="-Dquarkus.http.host=0.0.0.0 -Djava.util.logging.manager=org.jboss.logmanager.LogManager"

#ENV QUARKUS_LAUNCH_DEVMODE=true \
#    JAVA_ENABLE_DEBUG=true
# -Dquarkus.package.type=mutable-jar

# We make four distinct layers so if there are application changes the library layers can be re-used
COPY --chown=1001 target/quarkus-app/lib/ ${APP_HOME}/lib/
COPY --chown=1001 target/quarkus-app/*-run.jar ${APP_HOME}/app.jar
COPY --chown=1001 target/quarkus-app/app/ ${APP_HOME}/app/
COPY --chown=1001 target/quarkus-app/quarkus/ ${APP_HOME}/quarkus/

EXPOSE 8080
USER 1001

#ENTRYPOINT [ "/deployments/run-java.sh" ]

这是我的 docker-撰写：

version: '2'

services:
  zookeeper:
    container_name: zookeeper
    image: confluentinc/cp-zookeeper
    ports:
      - "2181:2181"
      - "2888:2888"
      - "3888:3888"
    environment:
      - ZOOKEEPER_CLIENT_PORT=2181
      - ZOOKEEPER_TICK_TIME=2000
    networks:
      - kafkastreams-network

  kafka:
    container_name: kafka
    image: confluentinc/cp-kafka
    ports:
      - "9092:9092"
    depends_on:
      - zookeeper
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      - KAFKA_INTER_BROKER_LISTENER_NAME=PLAINTEXT
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
      - KAFKA_AUTO_CREATE_TOPICS_ENABLE=true
      - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
      - KAFKA_TRANSACTION_STATE_LOG_MIN_ISR=1
      - KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR=1
      - KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS=100
    networks:
      - kafkastreams-network

  connect:
    container_name: connect
    image: debezium/connect
    ports:
      - "8083:8083"
    depends_on:
      - kafka
    environment:
      - BOOTSTRAP_SERVERS=kafka:29092
      - GROUP_ID=1
      - CONFIG_STORAGE_TOPIC=my_connect_configs
      - OFFSET_STORAGE_TOPIC=my_connect_offsets
      - STATUS_STORAGE_TOPIC=my_connect_statuses
    networks:
      - kafkastreams-network

  schema-registry:
    image: confluentinc/cp-schema-registry:5.5.0
    container_name: schema-registry
    ports:
      - "8081:8081"
    depends_on:
      - zookeeper
      - kafka
    environment:
      SCHEMA_REGISTRY_HOST_NAME: schema-registry
      SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: zookeeper:2181
    networks:
      - kafkastreams-network

  kafdrop:
    image: obsidiandynamics/kafdrop
    container_name: kafdrop
    restart: "no"
    ports:
      - "9001:9000"
    environment:
      KAFKA_BROKERCONNECT: kafka:29092
      JVM_OPTS: "-Xms16M -Xmx48M -Xss180K -XX:-TieredCompilation -XX:+UseStringDeduplication -noverify"
    depends_on:
      - kafka
      - schema-registry
    networks:
      - kafkastreams-network

  connector:
    image: connector
    depends_on:
      - zookeeper
      - kafka
      - connect
    environment:
      QUARKUS_KAFKA_STREAMS_BOOTSTRAP_SERVERS: kafka:9092
    networks:
      - kafkastreams-network

networks:
  kafkastreams-network:
    name: ks

我得到的错误是：

2021-08-05 11:52:35,433 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-admin-client-thread | connector-18d10d7d-b619-4715-a219-2557d70e0479-admin) [AdminClient clientId=connector-18d10d7d-b619-4715-a219-2557d70e0479-admin] Connection to node -1 (kafka/172.21.0.3:9092) could not be established. Broker may not be available.

我是否遗漏了 Dockerfile 或 docker compose 上的任何配置？

Answer 1

我发现有两个问题：

在我的 docker-compose 中，我不得不将属性 KAFKA_ADVERTISED_LISTENERS 更改为 PLAINTEXT://kafka:29092,PLAINTEXT_HOST://kafka:9092
在我的 quarkus application.properties 中，我有 2 个属性指向错误的位置：

quarkus.kafka-streams.bootstrap-servers=localhost:9092

quarkus.kafka-streams.application-server=localhost:9999

Quarkus 无法从内部连接到 kafka docker

Quarkus can't connect to kafka from inside docker

apache-kafka

docker

docker-compose

quarkus