Dynamodb 流运动 - 之间发现不完整的哈希范围

Question

我有来自 DynamoDB 的运动流。我正在通过 aws KCL sdk v1.14.0 处理它。我在日志中看到下面偶尔出现的错误。我还观察到，对于 dynamodb 租约 table 中的所有分片，startingHashKey 始终为 1，endingHashKey 始终为 0。知道为什么会这样吗？

Incomplete hash range found between {
  "leaseKey" : "shardId-00000001601992582645-697387df",
  "leaseOwner" : "eu01-stg01-vendor-service-74b4cd66b6-6hcfb:29dafc2f-c87f-4c70-b210-d38db7ebfc87",
  "leaseCounter" : 4080,
  "concurrencyToken" : null,
  "lastCounterIncrementNanos" : null,
  "checkpoint" : {
    "sequenceNumber" : "47515300000000010794885359",
    "subSequenceNumber" : 0,
    "shardEnd" : false
  },
  "pendingCheckpoint" : null,
  "ownerSwitchesSinceCheckpoint" : 0,
  "parentShardIds" : [ ],
  "childShardIds" : [ ],
  "hashKeyRange" : {
    "startingHashKey" : 0,
    "endingHashKey" : 1
  }
} and {
  "leaseKey" : "shardId-00000001602005637902-6358a08a",
  "leaseOwner" : "eu01-stg01-vendor-service-74b4cd66b6-6hcfb:29dafc2f-c87f-4c70-b210-d38db7ebfc87",
  "leaseCounter" : 397,
  "concurrencyToken" : null,
  "lastCounterIncrementNanos" : null,
  "checkpoint" : {
    "sequenceNumber" : "TRIM_HORIZON",
    "subSequenceNumber" : 0,
    "shardEnd" : false
  },
  "pendingCheckpoint" : null,
  "ownerSwitchesSinceCheckpoint" : 0,
  "parentShardIds" : [ "shardId-00000001601992582736-2c3fc0ff" ],
  "childShardIds" : [ ],
  "hashKeyRange" : {
    "startingHashKey" : 0,
    "endingHashKey" : 1
  }
}

Answer 1

我也遇到了这个错误，但流似乎工作正常。（我正在用它来监控数据变化）。

然后我挖了这个错误日志的源代码，发现部分代码是2020年6月22日添加的，

这是源代码：https://github.com/awslabs/amazon-kinesis-client/blob/6fbfc21ad7d6a2722491806a411d168304167f7f/src/main/java/com/amazonaws/services/kinesis/clientlibrary/lib/worker/PeriodicShardSyncManager.java#L298

这是怪：https://github.com/awslabs/amazon-kinesis-client/blame/6fbfc21ad7d6a2722491806a411d168304167f7f/src/main/java/com/amazonaws/services/kinesis/clientlibrary/lib/worker/PeriodicShardSyncManager.java#L298

这是提交：https://github.com/awslabs/amazon-kinesis-client/commit/3a88a60a4ee7bb8f969b36cb79e7665a7395b6ec

仅在最新版本：1.14.0

所以我回到一个版本使用 1.13.3 然后没有这个错误。

我已经在 GitHub 上提交了一个问题，看看错误的原因是什么。 https://github.com/awslabs/amazon-kinesis-client/issues/758

Dynamodb 流运动 - 之间发现不完整的哈希范围

Dynamodb stream kinesis - Incomplete hash range found between

amazon-dynamodb

amazon-kinesis

amazon-kcl