使用 Node.js 从 Cloud Functions 读取 Cloud Bigtable 需要 > 1500 毫秒

Question

我正在尝试使用 Node.JS 通过 Google Cloud Functions 读取 Cloud Bigtable 密钥并且我能够读取它，但是 Cloud Functions 执行时间超过 1500 毫秒。

我听说 Cloud Bigtable 在数据检索方面非常快，但在这种情况下并没有发生。

谁能帮我解决我这里做错的问题？

我尝试全局加载 Bigtable 库和对象：

/**
 * Responds to any HTTP request.
 *
 * @param {!express:Request} req HTTP request context.
 * @param {!express:Response} res HTTP response context.
 */

// Imports the Google Cloud client library
const Bigtable = require('@google-cloud/bigtable');

const TABLE_ID = '';
const COLUMN_FAMILY_ID = '';
const COLUMN_QUALIFIER = '';
const INSTANCE_ID = '';

// Creates a Bigtable client
const bigtable = new Bigtable();

// Connect to an existing instance:my-bigtable-instance
const instance = bigtable.instance(INSTANCE_ID);

// Connect to an existing table:my-table
const table = instance.table(TABLE_ID);

const filter = [{
  family: COLUMN_FAMILY_ID,
}, {
  column: COLUMN_QUALIFIER
}];

exports.helloWorld = (req, res) => {

    console.log("started");

    (async () => {
        try {

          var query_params = req.query;
          var rowkey = query_params.key;

          console.log("before query");

          const [singleRow] = await table.row(rowkey).get({filter});

          console.log("after query");

          res.status(200).send();

        } catch (err) {
            // Handle error performing the read operation
            console.error(`Error reading rows :`, err);
        }
    })();

};

我把控制台日志放在不同的点，查询前后的日志时间有1500ms左右的差距。

Answer 1

根据 documentation:

To get good performance from Cloud Bigtable, it's essential to design a schema that makes it possible to distribute reads and writes evenly across each table.

意思是，Bigtable 的性能在很大程度上取决于模式设计等，例如工作负载、每行单元格、每个集群的节点、磁盘等。不仅是从中访问它的环境（使用你的代码我访问了我的示例 Bigtable 表在 GCF 大约 750 毫秒和 Shell).

大约 4000 毫秒

此外，如果您希望正确测试 Bigtable 性能，建议在合适的情况下进行：

Use a production instance. A development instance will not give you an accurate sense of how a production instance performs under load.

Use at least 300 GB of data. Cloud Bigtable performs best with 1 TB or more of data. However, 300 GB of data is enough to provide reasonable results in a performance test on a 3-node cluster. On larger clusters, use at least 100 GB of data per node.

Stay below the recommended storage utilization per node. For details, see Storage utilization per node.

Before you test, run a heavy pre-test for several minutes. This step gives Cloud Bigtable a chance to balance data across your nodes based on the access patterns it observes.

Run your test for at least 10 minutes. This step lets Cloud Bigtable further optimize your data, and it helps ensure that you will test reads from disk as well as cached reads from memory.

使用 Node.js 从 Cloud Functions 读取 Cloud Bigtable 需要 > 1500 毫秒

Reading Cloud Bigtable from Cloud Functions using Node.js takes > 1500ms

google-cloud-bigtable

google-cloud-functions