Debezium 如何使用 Kafka Connect 正确注册 SqlServer 连接器 - 连接被拒绝

Question

问题：

如何使用 Kafka Connect 正确注册 SqlServer 连接器以连接到独立的 SQL 服务器实例？注意：我不是运行ning SQL 服务器 Docker.

错误：

Error: Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host 127.0.0.1, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.".

疑难解答：

在 dbo.Posts table 上启用了 CDC（见下文）
已验证 TCP/IP 已启用并在 SQL 服务器配置管理器的 SQL 服务器网络配置
禁用Windows防火墙
已在 SQL 服务器配置管理器中启用 SQL 服务器浏览器
已验证我可以通过以下方式远程登录：(127.0.0.1 1433; localhost 1433; lt-ls231 1433)

上下文：

我正在尝试使用 Docker 在 Windows 10 上设置 Debezium。我可以毫无问题地在 Powershell 中成功运行以下命令：

docker run -it --rm --name zookeeper -p 2181:2181 -p 2888:2888 -p 3888:3888 debezium/zookeeper:1.1 

docker run -it --rm --name kafka -p 9092:9092 --link zookeeper:zookeeper debezium/kafka:1.1 

docker run -it --rm --name connect -p 8083:8083 -e GROUP_ID=1 -e CONFIG_STORAGE_TOPIC=my_connect_configs -e  OFFSET_STORAGE_TOPIC=my_connect_offsets -e  STATUS_STORAGE_TOPIC=my_connect_statuses --link zookeeper:zookeeper --link kafka:kafka debezium/connect:1.1

当我尝试使用 Kafka-Connect connect 注册 SQL 服务器连接时出现错误（见上文），我运行以下命令：

Invoke-RestMethod -Method Post -Uri 'http://localhost:8083/connectors/' -Headers @{'Accept' = 'application/json'; 'Content-Type' = 'application/json'} -Body '{"name": "mssqlserver-localhost-testDb-connector",  "config": {"connector.class": "io.debezium.connector.sqlserver.SqlServerConnector", "database.hostname": "127.0.0.1", "database.port": "1433", "database.user": "svc_kafka", "database.password": "password", "database.dbname": "Posts", "database.server.name": "LT-LS231", "table.whitelist": "dbo.Posts", "database.history.kafka.bootstrap.servers": "kafka:9092", "database.history.kafka.topic": "dbhistory.LT-LS231" }}'

格式化后的JSON如下：

Invoke-RestMethod -Method Post -Uri 'http://localhost:8083/connectors/' -Headers @{'Accept' = 'application/json'; 'Content-Type' = 'application/json'} -Body '

{  
    "name": "mssqlserver-localhost-testDb-connector",  
    "config": {
        "connector.class": "io.debezium.connector.sqlserver.SqlServerConnector", 
        "database.hostname": "127.0.0.1", 
        "database.port": "1433", 
        "database.user": "svc_kafka", 
        "database.password": "password", 
        "database.dbname": "Posts", 
        "database.server.name": "LT-LS231", 
        "table.whitelist": "dbo.Posts", 
        "database.history.kafka.bootstrap.servers": "kafka:9092", 
        "database.history.kafka.topic": "dbhistory.LT-LS231" 
  }
}'

这是根据 Debezium 网站上提供的 JSON 建模的：https://debezium.io/documentation/reference/1.1/connectors/sqlserver.html#sqlserver-deploying-a-connector

Answer 1

问题出在对 localhost 的引用中。如果您运行在 Docker 容器中连接，那么 localhost 是容器而不是机器的本地主机。尝试在 docker0 接口或所有接口上公开 SQL 服务器，然后在注册请求中使用 docker0 IP。

Answer 2

non-production 环境的解决方案是使用 host.docker.internal。这可以在 Docker 的网站上找到：https://docs.docker.com/docker-for-windows/networking/。请参阅标题为 I WANT TO CONNECT FROM A CONTAINER TO A SERVICE ON THE HOST 的部分。我需要查看 Jiri 的回答中推荐的 Docker0。

正确的JSON如下：

{  
    "name": "LT-LS231-mssqlserver-connector",  
    "config": {
        "connector.class": "io.debezium.connector.sqlserver.SqlServerConnector", 
        "database.server.name": "staging", 
        "database.hostname": "host.docker.internal", 
        "database.port": "1433", 
        "database.user": "svc_kafka", 
        "database.password": "*********", 
        "database.dbname": "Test", 
        "table.whitelist": "dbo.Posts", 
        "database.history.kafka.bootstrap.servers": "kafka:9092", 
        "database.history.kafka.topic": "dbhistory.Posts" 
  }
}

Debezium 如何使用 Kafka Connect 正确注册 SqlServer 连接器 - 连接被拒绝

Debezium How do I correctly register the SqlServer connector with Kafka Connect - connection refused

sql-server

docker

apache-kafka

apache-kafka-connect

debezium