尝试在 bigtable 中模拟单元级 TTL，但整个列族数据正在被垃圾收集删除

Question

创建了一个具有以下规则的 table：因此，数据应在 1 秒后过期 (as per docs)

async function createTable() {
    console.log("Creating Table");
    const options = {
        families: [
            {
                name: 'payloads',
                rule: {
                    age: {
                        seconds: 1,
                        nanos: 0,
                    },
                },
            },
        ],
    };
    try {
        await table.create(options);
        console.log("Successfully Created Table");
    } catch (err) {
        console.error(`Error reading rows :`, err);
    }
}

然后像这样插入数据：

      const rowsToInsert = {
            key: "SUMEET",
            data: {
                payloads: {
                    '1': {
                        value: "NOTIFICATIONS_PAYLOAD_1",
                        timestamp: 1576500927000,
                    },
                    '2': {
                        value: "NOTIFICATIONS_PAYLOAD_2",
                        timestamp: 1576587327000,
                    },
                    '3': {
                        value: "NOTIFICATIONS_PAYLOAD_3",
                        timestamp: 1576673727000,
                    },
                    '4': {
                        value: "NOTIFICATIONS_PAYLOAD_4",
                        timestamp: 1576760127000,
                    },
                },
            },
        };

        await table.insert(rowsToInsert);

所以我添加了四个具有不同时间戳的单元格：

首先在写入数据时提前5分钟
第二个，提前 1 小时
第三名提前 1 天
第四，提前 2 天

这里的问题是当 m 读取数据时，整个列族数据都被删除了，但它应该只根据规则集删除第一个和第二个单元格

有什么遗漏或做错了什么吗？？

Answer 1

您有以毫秒为单位的时间戳。 Cloud Bigtable 需要微秒（参见 here）。基本上，将“000”添加到所有时间戳的末尾。

Answer 2

问题出在您的时间戳中，您很可能设置了过去的日期。我建议您使用 date methods from javascript 设置日期，而不是像现在这样手动设置日期。

我用下面的代码做了一些测试：

const Bigtable = require('@google-cloud/bigtable');
const bigtable = Bigtable();
const instance = bigtable.instance([instance]);
const table = instance.table([table]);
const now = new Date();


async function writeSimple() {
    var now = new Date(Date.now());
    var fiveMinutes = new Date(now.getTime() + 5 * 60000);
    var anHour = new Date(now.getTime() + 60 * 60000);
    var aDay = new Date(now.getTime() + 24 * 60 * 60000);
    var twoDays = new Date(now.getTime() + 48 * 60 * 60000);
  const rowsToInsert = {
            key: "SUMEET",
            data: {
                payloads: {
                    '1': {
                        value: "NOTIFICATIONS_PAYLOAD_1",
                        timestamp: now,
                    },
                    '2': {
                        value: "NOTIFICATIONS_PAYLOAD_2",
                        timestamp: fiveMinutes,
                    },
                    '3': {
                        value: "NOTIFICATIONS_PAYLOAD_3",
                        timestamp: anHour,
                    },
                    '4': {
                        value: "NOTIFICATIONS_PAYLOAD_4",
                        timestamp: aDay,
                    },
                    '5': {
                        value: "NOTIFICATIONS_PAYLOAD_5",
                        timestamp: twoDays,
                    },
                },
            },
        };

        await table.insert(rowsToInsert);
        console.log(`Successfully wrote row ${rowsToInsert.key}`);
}

并获得如下一行：

2019/12/17 16:53:33 -creds flag unset, will use gcloud credential
----------------------------------------
SUMEET
  payloads:1                               @ 2019/12/17-16:30:34.343000
    "NOTIFICATIONS_PAYLOAD_1"
  payloads:2                               @ 2019/12/17-16:35:34.343000
    "NOTIFICATIONS_PAYLOAD_2"
  payloads:3                               @ 2019/12/17-17:30:34.343000
    "NOTIFICATIONS_PAYLOAD_3"
  payloads:4                               @ 2019/12/18-16:30:34.343000
    "NOTIFICATIONS_PAYLOAD_4"
  payloads:5                               @ 2019/12/19-16:30:34.343000
    "NOTIFICATIONS_PAYLOAD_5"

在垃圾收集器通过后（大约 15 分钟后），我得到了您想要的结果：

2019/12/17 16:59:47 -creds flag unset, will use gcloud credential
----------------------------------------
SUMEET
  payloads:3                               @ 2019/12/17-17:30:34.343000
    "NOTIFICATIONS_PAYLOAD_3"
  payloads:4                               @ 2019/12/18-16:30:34.343000
    "NOTIFICATIONS_PAYLOAD_4"
  payloads:5                               @ 2019/12/19-16:30:34.343000
    "NOTIFICATIONS_PAYLOAD_5"

希望你觉得这有用！

尝试在 bigtable 中模拟单元级 TTL，但整个列族数据正在被垃圾收集删除

Trying to simulate cell level TTL in bigtable but whole column family data is getting removed by garbage collection

bigtable

node.js

google-cloud-platform

google-cloud-bigtable

google-cloud-functions