AWS DMS - InvalidParameterValueException) when calling the CreateReplicationTask operation: Replication Task Settings document error: Invalid json

Question

我正在尝试使用 AWS DMS cli 创建 DMS 复制任务。我正在尝试使用 json 文件传递任务设置，如下所示：

aws dms create-replication-task --replication-task-identifier dms-cli-test-replication-task-1 --source-endpoint-arn arn --target-endpoint-arn arn --replication-instance-arn arn --migration-type full-load-and-cdc --table-mappings ./table_mappings.json --replication-task-settings ./task_settings.json --region us-east-1

当我运行此命令时，抛出以下错误：

An error occurred (InvalidParameterValueException) when calling the CreateReplicationTask operation: Replication Task Settings document error: Invalid json

以下是我的 task_settings.json 文件的内容：

{
"TargetMetadata": {
"TargetSchema": "",
"SupportLobs": true,
"FullLobMode": true,
"LobChunkSize": 64,
"LimitedSizeLobMode": false,
"LobMaxSize": 0,
"InlineLobMaxSize": 0,
"LoadMaxFileSize": 0,
"ParallelLoadThreads": 0,
"ParallelLoadBufferSize": 0,
"BatchApplyEnabled": false,
"TaskRecoveryTableEnabled": false
},
"FullLoadSettings": {
"TargetTablePrepMode": "DO_NOTHING",
"CreatePkAfterFullLoad": false,
"StopTaskCachedChangesApplied": false,
"StopTaskCachedChangesNotApplied": false,
"MaxFullLoadSubTasks": 8,
"TransactionConsistencyTimeout": 600,
"CommitRate": 10000
},
"Logging": {
"EnableLogging": true,
"LogComponents": [
{
"Id": "SOURCE_UNLOAD",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "SOURCE_CAPTURE",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "TARGET_LOAD",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "TARGET_APPLY",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "TASK_MANAGER",
"Severity": "LOGGER_SEVERITY_DEFAULT"
}
]
},
"ControlTablesSettings": {
"historyTimeslotInMinutes": 5,
"ControlSchema": "",
"HistoryTimeslotInMinutes": 5,
"HistoryTableEnabled": true,
"SuspendedTablesTableEnabled": true,
"StatusTableEnabled": true
},
"StreamBufferSettings": {
"StreamBufferCount": 3,
"StreamBufferSizeInMB": 8,
"CtrlStreamBufferSizeInMB": 5
},
"ChangeProcessingDdlHandlingPolicy": {
"HandleSourceTableDropped": true,
"HandleSourceTableTruncated": true,
"HandleSourceTableAltered": true
},
"ErrorBehavior": {
"DataErrorPolicy": "LOG_ERROR",
"DataTruncationErrorPolicy": "LOG_ERROR",
"DataErrorEscalationPolicy": "SUSPEND_TABLE",
"DataErrorEscalationCount": 0,
"TableErrorPolicy": "SUSPEND_TABLE",
"TableErrorEscalationPolicy": "STOP_TASK",
"TableErrorEscalationCount": 0,
"RecoverableErrorCount": -1,
"RecoverableErrorInterval": 5,
"RecoverableErrorThrottling": true,
"RecoverableErrorThrottlingMax": 1800,
"ApplyErrorDeletePolicy": "IGNORE_RECORD",
"ApplyErrorInsertPolicy": "LOG_ERROR",
"ApplyErrorUpdatePolicy": "LOG_ERROR",
"ApplyErrorEscalationPolicy": "LOG_ERROR",
"ApplyErrorEscalationCount": 0,
"ApplyErrorFailOnTruncationDdl": false,
"FullLoadIgnoreConflicts": true,
"FailOnTransactionConsistencyBreached": false,
"FailOnNoTablesCaptured": false
},
"ChangeProcessingTuning": {
"BatchApplyPreserveTransaction": true,
"BatchApplyTimeoutMin": 1,
"BatchApplyTimeoutMax": 30,
"BatchApplyMemoryLimit": 500,
"BatchSplitSize": 0,
"MinTransactionSize": 1000,
"CommitTimeout": 1,
"MemoryLimitTotal": 1024,
"MemoryKeepTime": 60,
"StatementCacheSize": 50
},
"ValidationSettings": {
"EnableValidation": true,
"ValidationMode": "ROW_LEVEL",
"ThreadCount": 5,
"PartitionSize": 10000,
"FailureMaxCount": 10000,
"RecordFailureDelayInMinutes": 5,
"RecordSuspendDelayInMinutes": 30,
"MaxKeyColumnSize": 8096,
"TableFailureMaxCount": 1000,
"ValidationOnly": false,
"HandleCollationDiff": false,
"RecordFailureDelayLimitInMinutes": 0
},
"PostProcessingRules": null,
"CharacterSetSettings": null
}

我的 json 的格式没有任何问题。我不明白为什么它说无效 json。任何建议表示赞赏。谢谢。

Answer 1

这实际上不是您将设置文件或任何文件传递给aws-cli。它通常在路径前面加上 "file://"，同时将 JSON 文件传递给 cli 的任何参数。

尝试使用以下命令：

aws dms create-replication-task --replication-task-identifier dms-cli-test-replication-task-1 --source-endpoint-arn arn --target-endpoint-arn arn --replication-instance-arn arn --migration-type full-load-and-cdc --table-mappings file://table_mappings.json --replication-task-settings file://task_settings.json --region us-east-1

但是，我看不到这个特定参数将 json 文档作为设置文件。但我认为你应该试试。检查文档中的以下代码段。

--replication-task-identifier (string)

The replication task identifier.

Constraints:

Must contain from 1 to 255 alphanumeric characters or hyphens.
First character must be a letter.
Cannot end with a hyphen or contain two consecutive hyphens.

AWS DMS - InvalidParameterValueException) when calling the CreateReplicationTask operation: Replication Task Settings document error: Invalid json

AWS DMS - InvalidParameterValueException) when calling the CreateReplicationTask operation: Replication Task Settings document error: Invalid json

data-migration

database-migration

amazon-web-services

aws-dms