确认如何与 Google Cloud PubSubIO.Read 一起使用

How acknowledgement works with Google Cloud PubSubIO.Read

我查看了文档,但找不到 PubSubIO.Read 函数如何处理确认。具体来说,我感兴趣的是消息是一条一条地被确认,还是以微批处理的方式被确认。如果是后者,不知是否可以设置batch size

如有任何帮助,我们将不胜感激。

来自

Dataflow executes your code in bundles. After successful execution each bundle is committed to avoid re-execution on successfully processed elements. Bundles are not necessarily committed between every step in the pipeline. See the description of fusion optimization for details about when PCollections are materialized and committed.

For PubSub, messages that were read as part of a bundle will be acknowledged as part of committing the completion of that bundle. This means if you look at the PubSub read step, and any ParDos after it, these will be executed (and committed) together.

因此,消息的确认既不是一个一个的,也不是以可控的批次进行的。这取决于消息的处理如何以及何时提交到下游。