在 Eclipse 上使用 Dataflow Runner 的 Apache Beam MinimalWordcount 示例
Apache Beam MinimalWordcount example with Dataflow Runner on eclipse
我正在尝试 运行 使用来自 Eclipse 的 DataFlow运行ner 在 windows 上使用 MinimalWordCount -->运行 As Java 的 MinimalWordCount 示例来自 eclipse 的应用程序，它与使用我的 gcs bucket 的示例中的股票代码相同，但是我始终收到以下异常，有人可以让我知道这里的问题是什么吗？
- 我已经验证了存储桶名称是正确的。
- 我已经 运行 我的 Windows 机器上的 gcloud init。
Exception in thread "main" java.lang.RuntimeException: Failed to
construct instance from factory method
DataflowRunner#fromOptions(interface
org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:233)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:162)
at org.apache.beam.sdk.runners.PipelineRunner.fromOptions(PipelineRunner.java:56)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:135)
at com.dynaobject.apachebeamexperiment.MinimalWordCount.main(MinimalWordCount.java:77)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:222)
... 4 more
Caused by: java.lang.IllegalArgumentException: DataflowRunner requires gcpTempLocation, but failed to retrieve a value from
PipelineOptions
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:212)
... 9 more
Caused by: java.lang.IllegalArgumentException: Error constructing default value for gcpTempLocation: tempLocation is not a
valid GCS path, gs://tempxyz.
at org.apache.beam.sdk.options.GcpOptions$GcpTempLocationFactory.create(GcpOptions.java:219)
at org.apache.beam.sdk.options.GcpOptions$GcpTempLocationFactory.create(GcpOptions.java:205)
at org.apache.beam.sdk.options.ProxyInvocationHandler.returnDefaultHelper(ProxyInvocationHandler.java:575)
at org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:516)
at org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:155)
at com.sun.proxy.$Proxy15.getGcpTempLocation(Unknown Source)
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:210)
... 9 more
Caused by: java.lang.IllegalArgumentException: Output path does not exist or is not writeable: gs://tempxyz
at org.apache.beam.sdk.repackaged.com.google.common.base.Preconditions.checkArgument(Preconditions.java:191)
at org.apache.beam.sdk.util.GcsPathValidator.verifyPathIsAccessible(GcsPathValidator.java:78)
at org.apache.beam.sdk.util.GcsPathValidator.validateOutputFilePrefixSupported(GcsPathValidator.java:62)
at org.apache.beam.sdk.options.GcpOptions$GcpTempLocationFactory.create(GcpOptions.java:217)
... 15 more
文档中某处提到我需要运行以下内容。
gcloud auth application-default login
显然异常对解决这个问题没有帮助，我希望它能帮助其他人。
我正在尝试 运行 使用来自 Eclipse 的 DataFlow运行ner 在 windows 上使用 MinimalWordCount -->运行 As Java 的 MinimalWordCount 示例来自 eclipse 的应用程序，它与使用我的 gcs bucket 的示例中的股票代码相同，但是我始终收到以下异常，有人可以让我知道这里的问题是什么吗？
- 我已经验证了存储桶名称是正确的。
- 我已经 运行 我的 Windows 机器上的 gcloud init。
Exception in thread "main" java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions) at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:233) at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:162) at org.apache.beam.sdk.runners.PipelineRunner.fromOptions(PipelineRunner.java:56) at org.apache.beam.sdk.Pipeline.create(Pipeline.java:135) at com.dynaobject.apachebeamexperiment.MinimalWordCount.main(MinimalWordCount.java:77) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:222) ... 4 more Caused by: java.lang.IllegalArgumentException: DataflowRunner requires gcpTempLocation, but failed to retrieve a value from PipelineOptions at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:212) ... 9 more Caused by: java.lang.IllegalArgumentException: Error constructing default value for gcpTempLocation: tempLocation is not a valid GCS path, gs://tempxyz. at org.apache.beam.sdk.options.GcpOptions$GcpTempLocationFactory.create(GcpOptions.java:219) at org.apache.beam.sdk.options.GcpOptions$GcpTempLocationFactory.create(GcpOptions.java:205) at org.apache.beam.sdk.options.ProxyInvocationHandler.returnDefaultHelper(ProxyInvocationHandler.java:575) at org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:516) at org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:155) at com.sun.proxy.$Proxy15.getGcpTempLocation(Unknown Source) at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:210) ... 9 more Caused by: java.lang.IllegalArgumentException: Output path does not exist or is not writeable: gs://tempxyz at org.apache.beam.sdk.repackaged.com.google.common.base.Preconditions.checkArgument(Preconditions.java:191) at org.apache.beam.sdk.util.GcsPathValidator.verifyPathIsAccessible(GcsPathValidator.java:78) at org.apache.beam.sdk.util.GcsPathValidator.validateOutputFilePrefixSupported(GcsPathValidator.java:62) at org.apache.beam.sdk.options.GcpOptions$GcpTempLocationFactory.create(GcpOptions.java:217) ... 15 more
文档中某处提到我需要运行以下内容。
gcloud auth application-default login
显然异常对解决这个问题没有帮助，我希望它能帮助其他人。