DynamoDB 查询二级索引,如何定义索引
DynamoDB queries on secondary index, how to define the indexes
我一直在纠结这个问题,只是不清楚该怎么做。
我有一个简单的 table,我想在其中对多个列进行查询。据我了解,这意味着为每个要查询的列创建一个二级索引。我已经定义了 table——使用无服务器框架 serverless.yml
——并且收到了各种奇怪的错误消息。
当前的serverless.yml
定义是:
resources:
Resources:
MessagesDynamoDBTable:
Type: 'AWS::DynamoDB::Table'
Properties:
AttributeDefinitions:
- AttributeName: messageId
AttributeType: S
- AttributeName: room
AttributeType: S
- AttributeName: userId
AttributeType: S
KeySchema:
- AttributeName: messageId
KeyType: HASH
- AttributeName: userId
KeyType: RANGE
LocalSecondaryIndexes:
- IndexName: roomIndex
KeySchema:
- AttributeName: room
KeyType: HASH
Projection:
ProjectionType: "KEYS_ONLY"
- IndexName: userId
KeySchema:
- AttributeName: userId
KeyType: HASH
Projection:
ProjectionType: "KEYS_ONLY"
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
TableName: ${self:custom.tableName}
它应该类似于 Slack 服务 - 因此我想查询来自房间、用户等的条目。
根据我能找到的文档,这个定义很有意义。应该为索引中使用的列声明属性,我已经这样做了。 KeySchema - 我真的只需要 messageId 字段,但一条错误消息表明它需要一个 RANGE 索引,所以我将 userId 字段添加到 KeySchema 只是为了关闭它。根据我能够找到的文档,二级索引字段看起来是正确的。
使用此定义,我在尝试使用 serverless deploy
进行部署时遇到此错误
An error occurred: MessagesDynamoDBTable - Property AttributeDefinitions is inconsistent
with the KeySchema of the table and the secondary indexes.
我尝试了几种变体,也遇到了其他奇怪的错误。下面是一些,但我不记得相应的定义有什么变化。
An error occurred: MessagesDynamoDBTable - One or more parameter values were invalid:
Index KeySchema does not have a range key for index: userId (Service: AmazonDynamoDBv2; Status Code: 400;
Error Code: ValidationException; Request ID: 1KFA2IMASC12HRLLDPG753CU63VV4KQNSO5AEMVJF66Q9ASUAAJG).
An error occurred: MessagesDynamoDBTable - 1 validation error detected: Value '[com.amazonaws.dynamodb.v20120810.KeySchemaElement@aa4cdc91,
com.amazonaws.dynamodb.v20120810.KeySchemaElement@d2cd6f64, com.amazonaws.dynamodb.v20120810.KeySchemaElement@4d7c1f9,
com.amazonaws.dynamodb.v20120810.KeySchemaElement@d2cd6f64]' at 'keySchema' failed to satisfy constraint: Member must have length less
than or equal to 2 (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: BOVVBQ1F35VA18CCF3L5MSKS1FVV4KQNSO5AEMVJF66Q9ASUAAJG).
An error occurred: MessagesDynamoDBTable - Property AttributeDefinitions is inconsistent with the KeySchema
of the table and the secondary indexes.
An error occurred: MessagesDynamoDBTable - One or more parameter values were invalid: Index KeySchema does not have a range key for index:
userIdIndex (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: KFS63VSPKDUC60DV6U2V47UP27VV4KQNSO5AEMVJF66Q9ASUAAJG).
An error occurred: MessagesDynamoDBTable - One or more parameter values were invalid: Table KeySchema does not have a range key,
which is required when specifying a LocalSecondaryIndex (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: 86Q2JSPM6Q9UPNIEOVHALLIIQJVV4KQNSO5AEMVJF66Q9ASUAAJG).
它不起作用的原因是本地二级索引中的键必须与 table 具有相同的分区键。因此,在您的情况下,您的本地二级索引必须将 messageId
作为其 HASH
键,并且将 room
和 userId
作为其各自索引上的 RANGE
键。由于您的 table 已经由 (messageId, userId)
键入,因此您不需要 userId
Local Secondary Index。
此设置在技术上可行:
MessagesDynamoDBTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
- AttributeName: messageId
AttributeType: S
- AttributeName: room
AttributeType: S
- AttributeName: userId
AttributeType: S
KeySchema:
- AttributeName: messageId
KeyType: HASH
- AttributeName: userId
KeyType: RANGE
LocalSecondaryIndexes:
- IndexName: roomIndex
KeySchema:
- AttributeName: messageId
KeyType: HASH
- AttributeName: room
KeyType: RANGE
Projection:
ProjectionType: KEYS_ONLY
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
TableName: ${self:custom.tableName}
但是,如果您想按房间和用户进行查询,那么您可能希望采用不同的 table 设计。您尝试执行的操作最终会要求您始终使用 messageId
查询 table 作为查询的一部分,因为它是分区键。因此,您将无法仅通过 room
和 userId
进行查询。您可能想要的是 Global Secondary Indexes。在那种情况下,这将起作用:
MessagesDynamoDBTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
- AttributeName: messageId
AttributeType: S
- AttributeName: room
AttributeType: S
- AttributeName: userId
AttributeType: S
KeySchema:
- AttributeName: messageId
KeyType: HASH
GlobalSecondaryIndexes:
- IndexName: roomIndex
KeySchema:
- AttributeName: room
KeyType: HASH
Projection:
ProjectionType: KEYS_ONLY
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
- IndexName: userIndex
KeySchema:
- AttributeName: userId
KeyType: HASH
Projection:
ProjectionType: KEYS_ONLY
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
TableName: ${self:custom.tableName}
请注意,使 ProjectionType: KEYS_ONLY
意味着当您查询 roomIndex
或 userIndex
时,您将得到的只是 messageIds
- 然后您将不得不重新查询 table 与 messageIds
以获得其他属性。根据您的使用模式,您可能希望使用不同的 ProjectionType
。
我一直在纠结这个问题,只是不清楚该怎么做。
我有一个简单的 table,我想在其中对多个列进行查询。据我了解,这意味着为每个要查询的列创建一个二级索引。我已经定义了 table——使用无服务器框架 serverless.yml
——并且收到了各种奇怪的错误消息。
当前的serverless.yml
定义是:
resources:
Resources:
MessagesDynamoDBTable:
Type: 'AWS::DynamoDB::Table'
Properties:
AttributeDefinitions:
- AttributeName: messageId
AttributeType: S
- AttributeName: room
AttributeType: S
- AttributeName: userId
AttributeType: S
KeySchema:
- AttributeName: messageId
KeyType: HASH
- AttributeName: userId
KeyType: RANGE
LocalSecondaryIndexes:
- IndexName: roomIndex
KeySchema:
- AttributeName: room
KeyType: HASH
Projection:
ProjectionType: "KEYS_ONLY"
- IndexName: userId
KeySchema:
- AttributeName: userId
KeyType: HASH
Projection:
ProjectionType: "KEYS_ONLY"
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
TableName: ${self:custom.tableName}
它应该类似于 Slack 服务 - 因此我想查询来自房间、用户等的条目。
根据我能找到的文档,这个定义很有意义。应该为索引中使用的列声明属性,我已经这样做了。 KeySchema - 我真的只需要 messageId 字段,但一条错误消息表明它需要一个 RANGE 索引,所以我将 userId 字段添加到 KeySchema 只是为了关闭它。根据我能够找到的文档,二级索引字段看起来是正确的。
使用此定义,我在尝试使用 serverless deploy
An error occurred: MessagesDynamoDBTable - Property AttributeDefinitions is inconsistent
with the KeySchema of the table and the secondary indexes.
我尝试了几种变体,也遇到了其他奇怪的错误。下面是一些,但我不记得相应的定义有什么变化。
An error occurred: MessagesDynamoDBTable - One or more parameter values were invalid:
Index KeySchema does not have a range key for index: userId (Service: AmazonDynamoDBv2; Status Code: 400;
Error Code: ValidationException; Request ID: 1KFA2IMASC12HRLLDPG753CU63VV4KQNSO5AEMVJF66Q9ASUAAJG).
An error occurred: MessagesDynamoDBTable - 1 validation error detected: Value '[com.amazonaws.dynamodb.v20120810.KeySchemaElement@aa4cdc91,
com.amazonaws.dynamodb.v20120810.KeySchemaElement@d2cd6f64, com.amazonaws.dynamodb.v20120810.KeySchemaElement@4d7c1f9,
com.amazonaws.dynamodb.v20120810.KeySchemaElement@d2cd6f64]' at 'keySchema' failed to satisfy constraint: Member must have length less
than or equal to 2 (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: BOVVBQ1F35VA18CCF3L5MSKS1FVV4KQNSO5AEMVJF66Q9ASUAAJG).
An error occurred: MessagesDynamoDBTable - Property AttributeDefinitions is inconsistent with the KeySchema
of the table and the secondary indexes.
An error occurred: MessagesDynamoDBTable - One or more parameter values were invalid: Index KeySchema does not have a range key for index:
userIdIndex (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: KFS63VSPKDUC60DV6U2V47UP27VV4KQNSO5AEMVJF66Q9ASUAAJG).
An error occurred: MessagesDynamoDBTable - One or more parameter values were invalid: Table KeySchema does not have a range key,
which is required when specifying a LocalSecondaryIndex (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: 86Q2JSPM6Q9UPNIEOVHALLIIQJVV4KQNSO5AEMVJF66Q9ASUAAJG).
它不起作用的原因是本地二级索引中的键必须与 table 具有相同的分区键。因此,在您的情况下,您的本地二级索引必须将 messageId
作为其 HASH
键,并且将 room
和 userId
作为其各自索引上的 RANGE
键。由于您的 table 已经由 (messageId, userId)
键入,因此您不需要 userId
Local Secondary Index。
此设置在技术上可行:
MessagesDynamoDBTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
- AttributeName: messageId
AttributeType: S
- AttributeName: room
AttributeType: S
- AttributeName: userId
AttributeType: S
KeySchema:
- AttributeName: messageId
KeyType: HASH
- AttributeName: userId
KeyType: RANGE
LocalSecondaryIndexes:
- IndexName: roomIndex
KeySchema:
- AttributeName: messageId
KeyType: HASH
- AttributeName: room
KeyType: RANGE
Projection:
ProjectionType: KEYS_ONLY
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
TableName: ${self:custom.tableName}
但是,如果您想按房间和用户进行查询,那么您可能希望采用不同的 table 设计。您尝试执行的操作最终会要求您始终使用 messageId
查询 table 作为查询的一部分,因为它是分区键。因此,您将无法仅通过 room
和 userId
进行查询。您可能想要的是 Global Secondary Indexes。在那种情况下,这将起作用:
MessagesDynamoDBTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
- AttributeName: messageId
AttributeType: S
- AttributeName: room
AttributeType: S
- AttributeName: userId
AttributeType: S
KeySchema:
- AttributeName: messageId
KeyType: HASH
GlobalSecondaryIndexes:
- IndexName: roomIndex
KeySchema:
- AttributeName: room
KeyType: HASH
Projection:
ProjectionType: KEYS_ONLY
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
- IndexName: userIndex
KeySchema:
- AttributeName: userId
KeyType: HASH
Projection:
ProjectionType: KEYS_ONLY
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
TableName: ${self:custom.tableName}
请注意,使 ProjectionType: KEYS_ONLY
意味着当您查询 roomIndex
或 userIndex
时,您将得到的只是 messageIds
- 然后您将不得不重新查询 table 与 messageIds
以获得其他属性。根据您的使用模式,您可能希望使用不同的 ProjectionType
。