使用选定的列更改 table - debezium 抛出 ArrayIndexOutOfBoundsException

Question

CDC table 的列数少于来源 table。当 debezium 尝试在 Kafka 中创建事件时，失败并显示：

ArrayIndexOutOfBoundsException.

历史主题具有完整源 table 模式的快照。这是 debezium 的限制，即 CDC 模式不能与源模式不同吗？

This connector will be stopped.\n\tat io.debezium.connector.base.ChangeEventQueue.throwProducerFailureIfPresent(ChangeEventQueue.java:170)\n\tat 
io.debezium.connector.base.ChangeEventQueue.poll(ChangeEventQueue.java:151)\n\tat io.debezium.connector.sqlserver.SqlServerConnectorTask.poll(SqlServerConnectorTask.java:158)\n\tat 
org.apache.kafka.connect.runtime.WorkerSourceTask.poll(WorkerSourceTask.java:245)\n\tat org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:221)\n\tat 
org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)\n\tat org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)\n\tat 
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat
 java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat 
 java.base/java.lang.Thread.run(Thread.java:834)\nCaused by: java.lang.ArrayIndexOutOfBoundsException: Index 10 out of bounds for length 10\n\tat io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator(TableSchemaBuilder.java:210)\n\tat
  io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:135)\n\tat io.debezium.relational.RelationalChangeRecordEmitter.emitUpdateRecord(RelationalChangeRecordEmitter.java:89)\n\tat 
  io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:46)\n\tat 
  io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:125)\n\tat 
  io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.lambda$execute(SqlServerStreamingChangeEventSource.java:203)\n\tat 
  io.debezium.jdbc.JdbcConnection.prepareQuery(JdbcConnection.java:485)\n\tat 
  io.debezium.connector.sqlserver.SqlServerConnection.getChangesForTables(SqlServerConnection.java:143)\n\tat 
  io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.execute(SqlServerStreamingChangeEventSource.java:137)\n\tat 
  io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start[=11=](ChangeEventSourceCoordinator.java:86)\n\t... 5 more\n"}],"type":"source"}```

Answer 1

捕获 table 必须捕获源 table 中定义的所有列。

有一个 WIP PR https://github.com/debezium/debezium/pull/748 消除了这个限制。

使用选定的列更改 table - debezium 抛出 ArrayIndexOutOfBoundsException

Change table with selected columns - debezium throws ArrayIndexOutOfBoundsException

sql-server

cdc

apache-kafka-connect

debezium