使用 AWS CDK 创建 AWS DMS 任务

Question

我正在尝试使用 AWS CDK 创建 AWS DMS 任务。但我不知道从哪里开始。我找不到关于如何使用 CDK 创建 DMS 任务的好文档。我找到了关于这两个主题的文章，但找不到解决这个问题的文章 - 讨论如何使用 CDK 创建 DMS 任务。

任何人都可以指出解释这一点的正确文章或帮助我做到这一点吗？

P.S。 - 我已经用 dms maven 依赖项初始化了项目。我正在使用 JAVA.

谢谢

Answer 1

没有 CDK 构造来简化 DMS 的使用。因此，您必须使用 CloudFormation resources：CfnEndpoint、CfnReplicationTask 等

我提供以下示例来帮助您入门，但请注意，DMS CloudFormation 资源非常具有挑战性。

import * as cdk from '@aws-cdk/core';
import * as dms from '@aws-cdk/aws-dms';

export class DmsStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Create a subnet group that allows DMS to access your data
    const subnet = new dms.CfnReplicationSubnetGroup(this, 'SubnetGroup', {
      replicationSubnetGroupIdentifier: 'cdk-subnetgroup',
      replicationSubnetGroupDescription: 'Subnets that have access to my data source and target.',
      subnetIds: [ 'subnet-123', 'subnet-456' ],
    });

    // Launch an instance in the subnet group
    const instance = new dms.CfnReplicationInstance(this, 'Instance', {
      replicationInstanceIdentifier: 'cdk-instance',

      // Use the appropriate instance class: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.Types.html
      replicationInstanceClass: 'dms.t2.small',

      // Setup networking
      replicationSubnetGroupIdentifier: subnet.replicationSubnetGroupIdentifier,
      vpcSecurityGroupIds: [ 'sg-123' ],
    });

    // Create endpoints for your data, see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-dms-endpoint.html
    const source = new dms.CfnEndpoint(this, 'Source', {
      endpointIdentifier: 'cdk-source',
      endpointType: 'source',
      engineName: 'mysql',

      serverName: 'source.database.com',
      port: 3306,
      databaseName: 'database',
      username: 'dms-user',
      password: 'password-from-secret',
    });

    const target = new dms.CfnEndpoint(this, 'Target', {
      endpointIdentifier: 'cdk-target',
      endpointType: 'target',
      engineName: 's3',

      s3Settings: {
        bucketName: 'target-bucket'
      },
    });

    // Define the replication task
    const task = new dms.CfnReplicationTask(this, 'Task', {
      replicationInstanceArn: instance.ref,

      migrationType: 'full-load', // https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-dms-replicationtask.html#cfn-dms-replicationtask-migrationtype
      sourceEndpointArn: source.ref,
      targetEndpointArn: target.ref,
      tableMappings: JSON.stringify({
        "rules": [{
          "rule-type": "selection",
          "rule-id": "1",
          "rule-name": "1",
          "object-locator": {
            "schema-name": "%",
            "table-name": "%"
          },
          "rule-action": "include"
        }]
      })
    })
  }
}

Answer 2

只是对先前设置的补充 - 由于 DMS 上的一些更改 - 它不会等到 IAM 资源创建 - 因此将其添加为子网组资源的依赖项并添加对 subnetg 实例的依赖项，这应该会保存你有 2-3 个小时的时间来了解为什么它不工作但在筒仓的代码中工作....

import * as cdk from '@aws-cdk/core';
import * as dms from '@aws-cdk/aws-dms';
import { 
  ManagedPolicy, 
  Role, 
  ServicePrincipal, 
  PolicyStatement, 
  Effect 
} from '@aws-cdk/aws-iam';

import { App, Construct, Stack } from "@aws-cdk/core";

const app = new App();

app.synth()



export class DmsStack extends cdk.Stack {
  role: Role;
  public constructor(scope:cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);
    
    const dmsVPCServiceRole = new Role(this, 'dms-vpc-role', {
      assumedBy: new ServicePrincipal('dms.amazonaws.com'),
      roleName: 'dms-vpc-role'
    });

    // Add a policy to a Role
    dmsVPCServiceRole.addToPolicy(
      new PolicyStatement({
        effect: Effect.ALLOW,
        resources: ['*'],
        actions: [            
          'sts:AssumeRole',
        ]
      })
    );

    dmsVPCServiceRole.addToPolicy(
      new PolicyStatement({
        effect: Effect.ALLOW,
        resources: ['*'],
        actions: [            
          'dms:*',
        ]
      })
    );

    dmsVPCServiceRole.addToPolicy(
      new PolicyStatement({
        effect: Effect.ALLOW,
        resources: ['*'],
        actions: [            
          "kms:ListAliases", 
          "kms:DescribeKey"
        ]
      })
    );

    dmsVPCServiceRole.addToPolicy(
      new PolicyStatement({
        effect: Effect.ALLOW,
        resources: ['*'],
        actions: [            
          "iam:GetRole",
          "iam:PassRole",
          "iam:CreateRole",
          "iam:AttachRolePolicy"
        ]
      })
    );

    dmsVPCServiceRole.addToPolicy(
      new PolicyStatement({
        effect: Effect.ALLOW,
        resources: ['*'],
        actions: [       
          "ec2:CreateVpc", 
          "ec2:CreateSubnet",      
          "ec2:DescribeVpcs",
          "ec2:DescribeInternetGateways",
          "ec2:DescribeAvailabilityZones",
          "ec2:DescribeSubnets",
          "ec2:DescribeSecurityGroups",
          "ec2:ModifyNetworkInterfaceAttribute",
          "ec2:CreateNetworkInterface",
          "ec2:DeleteNetworkInterface"
        ]
      })
    );


    dmsVPCServiceRole.addToPolicy(
      new PolicyStatement({
        effect: Effect.ALLOW,
        resources: ['*'],
        actions: [            
          "logs:DescribeLogGroups",
          "logs:DescribeLogStreams",
          "logs:FilterLogEvents",
          "logs:GetLogEvents"
        ]
      })
    );

    dmsVPCServiceRole.addToPolicy(
      new PolicyStatement({
        effect: Effect.ALLOW,
        resources: ['arn:aws:s3:::BUCKETNAME/*'],
        actions: [            
          "s3:PutObject",
          "s3:DeleteObject",
          "s3:PutObjectTagging"
        ]
      })
    );

    dmsVPCServiceRole.addToPolicy(
      new PolicyStatement({
        effect: Effect.ALLOW,
        resources: ['arn:aws:s3:::BUCKETNAME'],
        actions: [            
          "s3:ListBucket"
        ]
      })
    );

    dmsVPCServiceRole.addToPolicy(
      new PolicyStatement({
        effect: Effect.ALLOW,
        resources: ['arn:aws:s3:::BUCKETNAME'],
        actions: [            
          "s3:GetBucketLocation"
        ]
      })
    );

    const dmsVpcManagementRolePolicy = ManagedPolicy.fromManagedPolicyArn(
      this, 
      'AmazonDMSVPCManagementRole', 
      'arn:aws:iam::aws:policy/service-role/AmazonDMSVPCManagementRole'
    );

    dmsVPCServiceRole.addManagedPolicy(dmsVpcManagementRolePolicy);





    
    // // Create a subnet group that allows DMS to access your data
    const subnet = new dms.CfnReplicationSubnetGroup(this, 'SubnetGroup', {
      replicationSubnetGroupIdentifier: 'cdk-subnetgroup',
      replicationSubnetGroupDescription: 'Subnets that have access to my data source and target.',
      subnetIds: ['subnet-01', 'subnet-02']
    });

    subnet.node.addDependency(dmsVPCServiceRole);



    const instance = new dms.CfnReplicationInstance(this, 'Instance', {
      replicationInstanceIdentifier: 'cdk-instance',

      // Use the appropriate instance class: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.Types.html
      replicationInstanceClass: 'dms.t2.small',

      // Setup networking
      replicationSubnetGroupIdentifier: subnet.replicationSubnetGroupIdentifier,
      vpcSecurityGroupIds: [ 'sg-041c1c796c1130121' ],
    });

    instance.node.addDependency(subnet)


  }
}

使用 AWS CDK 创建 AWS DMS 任务

Creating an AWS DMS task using AWS CDK

amazon-web-services

aws-dms

aws-cdk