如何从 GCP 中的 VM 实例修复 "Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java" 错误 运行 数据流模板？
How to fix "Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java" error running a Dataflow template from a VM instance in GCP?
我正在尝试在 VM 实例（OS："linux"，版本：“ 4.9.0-11-amd64", Distributor: Debian GNU/Linux 9.11 (stretch)) 从 Pub/Sub 订阅获取输入消息并将它们写入 BigQuery table（不修改模板暂时）。为此，我将 GitHub DataflowTemplates repo 克隆到 $HOME/opt/ 目录中的 Cloud Shell 中。按照 README 文档，我安装了 Java 8 和 Maven 3:
- Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) Maven 主页：/opt/maven
Java 版本: 1.8.0_232, 供应商: Oracle Corporation
Java 家：/usr/lib/jvm/java-8-openjdk-amd64/jre
默认语言环境：en_US，平台编码：UTF-8
OS 名称："linux"，版本：“4.9.0-11-amd64”，架构："amd64"，系列："unix"
构建整个项目后，这就是我试图从命令行执行以编译代码的内容：
sudo mvn -X compile exec:java -Dexec.mainClass=com.google.cloud.teleport.templates.PubSubToBigQuery -
Dexec.cleanupDaemonThreads=false -Dexec.args=" --project=real-demand --
stagingLocation=gs://rd_pubsub/staging --streaming=true"
这是我遇到的错误：
[WARNING]
java.lang.ClassNotFoundException: com.google.cloud.teleport.templates.PubSubToBigQuery.java
at java.net.URLClassLoader.findClass (URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass (ClassLoader.java:424)
at java.lang.ClassLoader.loadClass (ClassLoader.java:357)
at org.codehaus.mojo.exec.ExecJavaMojo.run (ExecJavaMojo.java:270)
at java.lang.Thread.run (Thread.java:748)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:21 min
[INFO] Finished at: 2019-11-26T10:47:57Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project google-cloud-teleport-java: An exception occured while executing the Java class. com.google.cloud.teleport.templates.PubSubToBigQuery.java -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project google-cloud-teleport-java: An exception occured while executing the Java class. com.google.cloud.teleport.templates.PubSubToBigQuery.java
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.MojoExecutionException: An exception occured while executing the Java class. com.google.cloud.teleport.templates.PubSubToBigQuery.java
at org.codehaus.mojo.exec.ExecJavaMojo.execute (ExecJavaMojo.java:339)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: java.lang.ClassNotFoundException: com.google.cloud.teleport.templates.PubSubToBigQuery.java
at java.net.URLClassLoader.findClass (URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass (ClassLoader.java:424)
at java.lang.ClassLoader.loadClass (ClassLoader.java:357)
at org.codehaus.mojo.exec.ExecJavaMojo.run (ExecJavaMojo.java:270)
at java.lang.Thread.run (Thread.java:748)
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
在 or this discussion discussion之后，我还在pom.xml中添加了XML的这段：
<properties>
<testNgMainClass>defaultMainClass</testNgMainClass>
<args>defaultArgs</args>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<configuration>
<classpathScope>test</classpathScope>
<mainClass>${testNgMainClass}</mainClass>
<arguments>${args}</arguments>
</configuration>
<executions>
<execution>
<id>run-selenium</id>
<phase>integration-test</phase>
<goals><goal>java</goal></goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
但是还是不行。
我看到您没有指定执行 template 所需的所有参数，而且正如您提到的，您没有修改文件。
- inputTopic=项目/${PROJECT_ID}/topics/input-topic-name
- outputTableSpec=${PROJECT_ID}:dataset-id.output-table
- outputDeadletterTable=${PROJECT_ID}:dataset-id.deadletter-table
命令应如下所示：
mvn compile exec:java -Dexec.mainClass=com.google.cloud.teleport.templates.PubSubToBigQuery -Dexec.cleanupDaemonThreads=false \
-Dexec.args=" --project=PROJECT_ID \
--stagingLocation=gs://bucket-name/dataflow \
--streaming=true \
--inputTopic=projects/PROJECT_ID/topics/input-topic-name \
--outputTableSpec=PROJECT_ID:dataset-id.deadletter-table \
--outputDeadletterTable=PROJECT_ID:dataset-id.deadletter-table"
另一方面，我建议将模板部署到 GCP 中并使用 gcloud 命令执行，按照 PubSubToBigQuery.java 文件中给出的 instructions
要克隆、编译和 运行 需要一个数据流模板，在您的 GCP 项目中启用所有必要的 API：
- 数据流
- 计算引擎
- Stackdriver 日志记录
- 云存储
- 云存储JSON
- BigQuery
- PubSub
为此，您可以单击此助手 link:
我正在尝试在 VM 实例（OS："linux"，版本：“ 4.9.0-11-amd64", Distributor: Debian GNU/Linux 9.11 (stretch)) 从 Pub/Sub 订阅获取输入消息并将它们写入 BigQuery table（不修改模板暂时）。为此，我将 GitHub DataflowTemplates repo 克隆到 $HOME/opt/ 目录中的 Cloud Shell 中。按照 README 文档，我安装了 Java 8 和 Maven 3:
- Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) Maven 主页：/opt/maven Java 版本: 1.8.0_232, 供应商: Oracle Corporation Java 家：/usr/lib/jvm/java-8-openjdk-amd64/jre 默认语言环境：en_US，平台编码：UTF-8 OS 名称："linux"，版本：“4.9.0-11-amd64”，架构："amd64"，系列："unix"
构建整个项目后，这就是我试图从命令行执行以编译代码的内容：
sudo mvn -X compile exec:java -Dexec.mainClass=com.google.cloud.teleport.templates.PubSubToBigQuery -
Dexec.cleanupDaemonThreads=false -Dexec.args=" --project=real-demand --
stagingLocation=gs://rd_pubsub/staging --streaming=true"
这是我遇到的错误：
[WARNING]
java.lang.ClassNotFoundException: com.google.cloud.teleport.templates.PubSubToBigQuery.java
at java.net.URLClassLoader.findClass (URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass (ClassLoader.java:424)
at java.lang.ClassLoader.loadClass (ClassLoader.java:357)
at org.codehaus.mojo.exec.ExecJavaMojo.run (ExecJavaMojo.java:270)
at java.lang.Thread.run (Thread.java:748)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:21 min
[INFO] Finished at: 2019-11-26T10:47:57Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project google-cloud-teleport-java: An exception occured while executing the Java class. com.google.cloud.teleport.templates.PubSubToBigQuery.java -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project google-cloud-teleport-java: An exception occured while executing the Java class. com.google.cloud.teleport.templates.PubSubToBigQuery.java
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.MojoExecutionException: An exception occured while executing the Java class. com.google.cloud.teleport.templates.PubSubToBigQuery.java
at org.codehaus.mojo.exec.ExecJavaMojo.execute (ExecJavaMojo.java:339)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: java.lang.ClassNotFoundException: com.google.cloud.teleport.templates.PubSubToBigQuery.java
at java.net.URLClassLoader.findClass (URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass (ClassLoader.java:424)
at java.lang.ClassLoader.loadClass (ClassLoader.java:357)
at org.codehaus.mojo.exec.ExecJavaMojo.run (ExecJavaMojo.java:270)
at java.lang.Thread.run (Thread.java:748)
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
在
<properties>
<testNgMainClass>defaultMainClass</testNgMainClass>
<args>defaultArgs</args>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<configuration>
<classpathScope>test</classpathScope>
<mainClass>${testNgMainClass}</mainClass>
<arguments>${args}</arguments>
</configuration>
<executions>
<execution>
<id>run-selenium</id>
<phase>integration-test</phase>
<goals><goal>java</goal></goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
但是还是不行。
我看到您没有指定执行 template 所需的所有参数，而且正如您提到的，您没有修改文件。
- inputTopic=项目/${PROJECT_ID}/topics/input-topic-name
- outputTableSpec=${PROJECT_ID}:dataset-id.output-table
- outputDeadletterTable=${PROJECT_ID}:dataset-id.deadletter-table
命令应如下所示：
mvn compile exec:java -Dexec.mainClass=com.google.cloud.teleport.templates.PubSubToBigQuery -Dexec.cleanupDaemonThreads=false \
-Dexec.args=" --project=PROJECT_ID \
--stagingLocation=gs://bucket-name/dataflow \
--streaming=true \
--inputTopic=projects/PROJECT_ID/topics/input-topic-name \
--outputTableSpec=PROJECT_ID:dataset-id.deadletter-table \
--outputDeadletterTable=PROJECT_ID:dataset-id.deadletter-table"
另一方面，我建议将模板部署到 GCP 中并使用 gcloud 命令执行，按照 PubSubToBigQuery.java 文件中给出的 instructions
要克隆、编译和 运行 需要一个数据流模板，在您的 GCP 项目中启用所有必要的 API：
- 数据流
- 计算引擎
- Stackdriver 日志记录
- 云存储
- 云存储JSON
- BigQuery
- PubSub
为此，您可以单击此助手 link: