kafka connect 的 docker compose 文件中的 env 变量应该是什么以导出 jmx 指标
What should be the env variables in the docker compose file of kafka connect to export jmx metrics
PFB 我用过的compose文件配置
version: '2'
services:
kafka-connect:
image: confluentinc/cp-kafka-connect:5.5.0
hostname: kafka-connect
ports:
- "8083:8083"
- "8000:8000"
environment:
CONNECT_BOOTSTRAP_SERVERS: "kafk_server_host_name:9092"
CONNECT_REST_PORT: 8083
CONNECT_GROUP_ID: compose-connect-group
CONNECT_CONFIG_STORAGE_TOPIC: docker-connect-configs
CONNECT_OFFSET_STORAGE_TOPIC: docker-connect-offsets
CONNECT_STATUS_STORAGE_TOPIC: docker-connect-status
CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
CONNECT_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
CONNECT_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
CONNECT_REST_ADVERTISED_HOST_NAME: "kafka-connect"
CONNECT_LOG4J_ROOT_LOGLEVEL: "INFO"
CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: "1"
CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: "1"
CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: "1"
KAFKA_JMX_OPTS: "-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -
Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=127.0.0.1 -
Dcom.sun.management.jmxremote.rmi.port=8000"
CONNECT_PLUGIN_PATH: '/usr/share/java,/etc/kafka-connect/jars'
volumes:
- ./libs:/etc/kafka-connect/jars/
在 docker 中启动 kafka connect 后,当我尝试使用 jconsole 连接到 localhost 8000 时,出现以下错误:
java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
java.io.EOFException
请帮助解决错误。我不确定我使用的环境变量是否正确。
我还有一个问题:在 kafka 中公开 jmx 端口是否也为 kafka connect 提供指标,还是应该分别从 kafka 和 kafka connect 获取指标?
如 Robin Moffatt 提供的资源所述
https://rmoff.net/2018/09/17/accessing-kafka-docker-containers-jmx-from-host/
我提到了两个变量 KAFKA_JMX_HOSTNAME 和 KAFKA_JMX_PORT 并特别使用了 9010 端口,因为其他端口给我问题中给出的 jrmf 错误
PFB 我用过的compose文件配置
version: '2'
services:
kafka-connect:
image: confluentinc/cp-kafka-connect:5.5.0
hostname: kafka-connect
ports:
- "8083:8083"
- "8000:8000"
environment:
CONNECT_BOOTSTRAP_SERVERS: "kafk_server_host_name:9092"
CONNECT_REST_PORT: 8083
CONNECT_GROUP_ID: compose-connect-group
CONNECT_CONFIG_STORAGE_TOPIC: docker-connect-configs
CONNECT_OFFSET_STORAGE_TOPIC: docker-connect-offsets
CONNECT_STATUS_STORAGE_TOPIC: docker-connect-status
CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
CONNECT_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
CONNECT_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
CONNECT_REST_ADVERTISED_HOST_NAME: "kafka-connect"
CONNECT_LOG4J_ROOT_LOGLEVEL: "INFO"
CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: "1"
CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: "1"
CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: "1"
KAFKA_JMX_OPTS: "-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -
Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=127.0.0.1 -
Dcom.sun.management.jmxremote.rmi.port=8000"
CONNECT_PLUGIN_PATH: '/usr/share/java,/etc/kafka-connect/jars'
volumes:
- ./libs:/etc/kafka-connect/jars/
在 docker 中启动 kafka connect 后,当我尝试使用 jconsole 连接到 localhost 8000 时,出现以下错误:
java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
java.io.EOFException
请帮助解决错误。我不确定我使用的环境变量是否正确。
我还有一个问题:在 kafka 中公开 jmx 端口是否也为 kafka connect 提供指标,还是应该分别从 kafka 和 kafka connect 获取指标?
如 Robin Moffatt 提供的资源所述
https://rmoff.net/2018/09/17/accessing-kafka-docker-containers-jmx-from-host/
我提到了两个变量 KAFKA_JMX_HOSTNAME 和 KAFKA_JMX_PORT 并特别使用了 9010 端口,因为其他端口给我问题中给出的 jrmf 错误