当我们在运行时增加碎片时,Kinesis 如何保证消息顺序?
How Kinesis guarantee the messages order while we increase the Shards in runtime?
我是 Kinesis 的新手,当我阅读 Kinesis 文档时,我意识到具有相同分区键的消息将映射到相同的分片以保证顺序。这完全有道理。但是,我想知道如果我们在运行时增加主题碎片的数量,具有相同分区键的新消息是否会像以前一样散列到相同的碎片(旧的)?
如果是这样,如果所有消息都提供了分区键,那么其中 none 将被映射到新的分片会怎么样?这对我来说没有意义。
如果不是,那么Kinesis如何保证具有相同分区键的消息的顺序?
了解 Kinesis 在 resharding:
时的作用很重要
There are two types of resharding operations: shard split and shard merge. In a shard split, you divide a single shard into two shards. In a shard merge, you combine two shards into a single shard. Resharding is always pairwise in the sense that you cannot split into more than two shards in a single operation, and you cannot merge more than two shards in a single operation. The shard or pair of shards that the resharding operation acts on are referred to as parent shards. The shard or pair of shards that result from the resharding operation are referred to as child shards.
当您增加运动流的大小时,您将split shards
When you split the shard, you specify a value in this range. That hash key value and all higher hash key values are distributed to one of the child shards. All the lower hash key values are distributed to the other child shard.
实际上 AWS 仅保证分片级别的顺序,因此在拆分后您将需要从两个分片读取并在消费者端进行排序,请参阅 here。
我是 Kinesis 的新手,当我阅读 Kinesis 文档时,我意识到具有相同分区键的消息将映射到相同的分片以保证顺序。这完全有道理。但是,我想知道如果我们在运行时增加主题碎片的数量,具有相同分区键的新消息是否会像以前一样散列到相同的碎片(旧的)?
如果是这样,如果所有消息都提供了分区键,那么其中 none 将被映射到新的分片会怎么样?这对我来说没有意义。
如果不是,那么Kinesis如何保证具有相同分区键的消息的顺序?
了解 Kinesis 在 resharding:
时的作用很重要There are two types of resharding operations: shard split and shard merge. In a shard split, you divide a single shard into two shards. In a shard merge, you combine two shards into a single shard. Resharding is always pairwise in the sense that you cannot split into more than two shards in a single operation, and you cannot merge more than two shards in a single operation. The shard or pair of shards that the resharding operation acts on are referred to as parent shards. The shard or pair of shards that result from the resharding operation are referred to as child shards.
当您增加运动流的大小时,您将split shards
When you split the shard, you specify a value in this range. That hash key value and all higher hash key values are distributed to one of the child shards. All the lower hash key values are distributed to the other child shard.
实际上 AWS 仅保证分片级别的顺序,因此在拆分后您将需要从两个分片读取并在消费者端进行排序,请参阅 here。