Kafka 连接 CockroachDB

Question

我正在尝试使用 CockroachDB (v2.0.6) 作为我的 Kafka 主题之一的接收器。

我找不到任何专门用于 CockroachDB 的 Kafka 连接器，所以我决定使用 Confluent 的 jdbc 接收器连接器，因为 CockroachDB 支持 postgreSQL 语法。

我在 Kafka Connect 上使用的连接字符串如下

"connection.url": "jdbc:postgresql://roach1:26257/mydb?sslmode=disable"

这基本上是我对现有工作的 Postgres 接收器连接器所做的唯一更改。

不幸的是我无法让它工作，因为连接器出现故障错误

Caused by: org.apache.kafka.connect.errors.ConnectException: java.sql.SQLException: org.postgresql.util.PSQLException: ERROR: syntax error at or near "."
  Detail: source SQL:
SELECT NULL AS TABLE_CAT, n.nspname AS TABLE_SCHEM,   ct.relname AS TABLE_NAME, a.attname AS COLUMN_NAME,   (i.keys).n AS KEY_SEQ, ci.relname AS PK_NAME FROM pg_catalog.pg_class ct   JOIN pg_catalog.pg_attribute a ON (ct.oid = a.attrelid)   JOIN pg_catalog.pg_namespace n ON (ct.relnamespace = n.oid)   JOIN (SELECT i.indexrelid, i.indrelid, i.indisprimary,              information_schema._pg_expandarray(i.indkey) AS keys         FROM pg_catalog.pg_index i) i     ON (a.attnum = (i.keys).x AND a.attrelid = i.indrelid)   JOIN pg_catalog.pg_class ci ON (ci.oid = i.indexrelid) WHERE true  AND ct.relname = 'my_topic' AND i.indisprimary  ORDER BY table_name, pk_name, key_seq
    at io.confluent.connect.jdbc.sink.JdbcSinkTask.put(JdbcSinkTask.java:88)
    at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:564)
    ... 10 more
Caused by: java.sql.SQLException: org.postgresql.util.PSQLException: ERROR: syntax error at or near "."
  Detail: source SQL:
SELECT NULL AS TABLE_CAT, n.nspname AS TABLE_SCHEM,   ct.relname AS TABLE_NAME, a.attname AS COLUMN_NAME,   (i.keys).n AS KEY_SEQ, ci.relname AS PK_NAME FROM pg_catalog.pg_class ct   JOIN pg_catalog.pg_attribute a ON (ct.oid = a.attrelid)   JOIN pg_catalog.pg_namespace n ON (ct.relnamespace = n.oid)   JOIN (SELECT i.indexrelid, i.indrelid, i.indisprimary,              information_schema._pg_expandarray(i.indkey) AS keys         FROM pg_catalog.pg_index i) i     ON (a.attnum = (i.keys).x AND a.attrelid = i.indrelid)   JOIN pg_catalog.pg_class ci ON (ci.oid = i.indexrelid) WHERE true  AND ct.relname = 'collect_flow_tracking' AND i.indisprimary  ORDER BY table_name, pk_name, key_seq

所以我的问题是，有没有人成功地将 Kafka Connect 与 CockroachDB 结合使用？还有人对这个错误（是什么原因造成的）有任何指示以及如何规避它并使它起作用吗？

Answer 1

这里是 CockroachDB 私信。看起来问题是由 Kafka Connect Postgres 连接器执行的不受支持的数据库自省查询。好消息是 CockroachDB 2.1 似乎支持这个特定的查询。您可以使用 the latest CockroachDB beta 再试一次吗？

Kafka 连接 CockroachDB

Kafka Connect with CockroachDB

apache-kafka-connect

cockroachdb