使用 `add_value_provider_argument` 和 apache beam io 函数创建 "RuntimeValueProviderError"

Question

我正在尝试创建一个将输入参数作为 RuntimeValue 的数据流模板。按照 the docs

中的示例

import re

import apache_beam as beam
from apache_beam.io import ReadFromText
from apache_beam.io import WriteToText
from apache_beam.options.pipeline_options import PipelineOptions

# [START example_wordcount_templated]
class WordcountTemplatedOptions(PipelineOptions):
  @classmethod
  def _add_argparse_args(cls, parser):
    # Use add_value_provider_argument for arguments to be templatable
    # Use add_argument as usual for non-templatable arguments
    parser.add_value_provider_argument(
        '--input', help='Path of the file to read from')
    parser.add_argument(
        '--output', required=True, help='Output file to write results to.')

pipeline_options = PipelineOptions(['--output', 'some/output_path'])
with beam.Pipeline(options=pipeline_options) as p:

  wordcount_options = pipeline_options.view_as(WordcountTemplatedOptions)
  lines = p | 'Read' >> ReadFromText(wordcount_options.input)
# [END example_wordcount_templated]

（直接取自 the official snippets）在尝试使用以下命令（已填写具体信息）创建模板时出现以下错误：

 python -m examples.mymodule \
    --runner DataflowRunner \
    --project YOUR_PROJECT_ID \
    --staging_location gs://YOUR_BUCKET_NAME/staging \
    --temp_location gs://YOUR_BUCKET_NAME/temp \
    --template_location gs://YOUR_BUCKET_NAME/templates/YOUR_TEMPLATE_NAME

  File "lib/python3.7/site-packages/apache_beam/options/value_provider.py", line 139, in _f                                                              
    raise error.RuntimeValueProviderError('%s not accessible' % obj)
apache_beam.error.RuntimeValueProviderError: RuntimeValueProvider(option: input, type: str, default_value: None) not
 accessible

docs 还指出：

Some I/O connectors contain methods that accept ValueProvider objects. To determine support for I/O connectors and their methods, see the API reference documentation for the connector. The following I/O connectors accept runtime parameters:

File-based IOs: textio, avroio, tfrecordio

我不确定示例代码为何出错。有人可以帮帮我吗？

我正在使用的物有所值：

apache-beam = {extras = ["gcp"], version = "^2.19.0"}

Answer 1

这已在 beam 0.20.0 中修复，自 4/15/2020 发布。

将 beam.io 操作（例如 beam.io.ReadFromText(wordcount_options.input) 与 RuntimeValueProvider 一起使用很简单。如果你遇到我在问题中遇到的相同错误，请尝试升级你的 beam 依赖版本。

相关的 JIRA 问题是：BEAM-9218
相关 PR 是：beam#10728

使用 `add_value_provider_argument` 和 apache beam io 函数创建 "RuntimeValueProviderError"

Using `add_value_provider_argument` with apache beam io functions creates "RuntimeValueProviderError"

python

apache-beam

apache-beam-io