AWS RDS 只读副本：如何避免复制少数表和列

Question

我正在使用 RDS MariaDB，并且需要拥有一个可用于分析的从数据库。我正在寻找 AWS Read Replica，但问题是它没有为我提供跳过包含一些敏感信息的少数表和列的复制的规定，我们不希望拥有包含该信息的从数据库。

我可以使用 AWS Read Replica 跳过少数表和列的复制吗？或者在 AWS Read Replica 中，我可以编写 AWS Lambda 来从副本数据库中删除此信息吗？

Answer 1

RDS 只读副本是只读的。您不能直接修改此副本。

更好的方法是创建适当的 table 和列级权限，以便分析用户无法访问某些 table。

MySQL Table and Column Level Permissions

Answer 2

AWS Database Migration Service 适用于此用例：

您可以使用 AWS Database Migration Service 在异构源数据库和目标数据库之间建立持续的连续复制。
AWS DMS 支持将 MariaDB 用于源数据库和目标数据库。
AWS DMS 支持从源数据库中按table 选择。
AWS DMS 支持在转换到目标数据库期间删除列。

根据常见问题：

Q. In addition to one-time data migration, can I use AWS Database Migration Service for continuous data replication?

Yes, you can use AWS Database Migration Service for both one-time data migration into RDS and EC2-based databases as well as for continuous data replication. AWS Database Migration Service will capture changes on the source database and apply them in a transactionally-consistent way to the target. Continuous replication can be done from your data center to the databases in AWS or in the reverse, replicating to a database in your datacenter from a database in AWS. Ongoing continuous replication can also be done between homogenous or heterogeneous databases. For ongoing replication it would be preferable to use Multi-AZ for high-availability.

如果您不熟悉 AWS DMS，请查看官方文档“Getting Started”页面，然后继续下面的内容。

使用与源相同的模式创建目标数据库，除了这几个您不想填充的 table 和列。

将源数据库和目标数据库与 DMS 相关联后，您需要创建一个任务来执行迁移，并使用特定的 table 映射来排除 table 并删除列.

排除表格

创建一个带有规则操作排除的 selection rule，并指定要排除的 table。迁移所有 tables 的文档示例，但 tablenames 以 DMS%:

开头的除外

{
    "rules": [
        {
            "rule-type": "selection",
            "rule-id": "1",
            "rule-name": "1",
            "object-locator": {
                "schema-name": "Test",
                "table-name": "%"
            },
            "rule-action": "include"
        },
        {
            "rule-type": "selection",
            "rule-id": "2",
            "rule-name": "2",
            "object-locator": {
                "schema-name": "Test",
                "table-name": "DMS%"
            },
            "rule-action": "exclude"
        }
    ]
}

删除列

使用 rule-action remove 和 rule-target any 创建一个 transformation rule 并指定要删除的列的架构、table 和列名。以下示例也来自文档，它删除了 test.Actor table 中以字符 col:

开头的所有列

{
    "rules": [{
        "rule-type": "selection",
        "rule-id": "1",
        "rule-name": "1",
        "object-locator": {
            "schema-name": "test",
            "table-name": "%"
        },
        "rule-action": "include"
    }, {
        "rule-type": "transformation",
        "rule-id": "2",
        "rule-name": "2",
        "rule-action": "remove-column",
        "rule-target": "column",
        "object-locator": {
            "schema-name": "test",
            "table-name": "Actor",
            "column-name": "col%"
        }
    }]
 }

AWS RDS 只读副本：如何避免复制少数表和列

AWS RDS Read Replica: How can I avoid replication of few tables and columns

java

amazon-web-services

amazon-rds

mariadb

aws-lambda

排除表格

删除列

进一步阅读