oozie 4.3.0 构建失败
oozie 4.3.0 build failed
我需要使用 sqoop-1.4.6 参数将配置单元 table 导出到 mysql，其中配置单元 table 有两个分区。但是我发现我的 hadoop 集群上的 oozie 共享库只支持 sqoop-1.4.3。所以我尝试构建和创建符合我们版本规范的共享库。
我修改了oozie pom.xml作为我们的集群主节点的情况，但是如果我做了以下两个更改，构建失败：
- 如果我将配置单元版本 Ln 92 从
<hive.version>1.2.0</hive.version> 更改为 <hive.version>2.1.0</hive.version>，我会收到以下错误：
Blockquote
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Oozie Main .................................. SUCCESS [ 0.991 s]
[INFO] Apache Oozie Hadoop Utils hadoop-2-4.3.0 ........... SUCCESS [ 1.843 s]
[INFO] Apache Oozie Hadoop Distcp hadoop-2-4.3.0 .......... SUCCESS [ 0.215 s]
[INFO] Apache Oozie Hadoop Auth hadoop-2-4.3.0 Test ....... SUCCESS [ 0.432 s]
[INFO] Apache Oozie Hadoop Libs ........................... SUCCESS [ 0.017 s]
[INFO] Apache Oozie Client ................................ SUCCESS [ 12.492 s]
[INFO] Apache Oozie Share Lib Oozie ....................... SUCCESS [ 1.868 s]
[INFO] Apache Oozie Share Lib HCatalog .................... SUCCESS [ 8.001 s]
[INFO] Apache Oozie Share Lib Distcp ...................... SUCCESS [ 0.733 s]
[INFO] Apache Oozie Core .................................. FAILURE [ 16.672 s]
[INFO] Apache Oozie Share Lib Streaming ................... SKIPPED
[INFO] Apache Oozie Share Lib Pig ......................... SKIPPED
[INFO] Apache Oozie Share Lib Hive ........................ SKIPPED
[INFO] Apache Oozie Share Lib Hive 2 ...................... SKIPPED
[INFO] Apache Oozie Share Lib Sqoop ....................... SKIPPED
[INFO] Apache Oozie Examples .............................. SKIPPED
[INFO] Apache Oozie Share Lib Spark ....................... SKIPPED
[INFO] Apache Oozie Share Lib ............................. SKIPPED
[INFO] Apache Oozie Docs .................................. SKIPPED
[INFO] Apache Oozie WebApp ................................ SKIPPED
[INFO] Apache Oozie Tools ................................. SKIPPED
[INFO] Apache Oozie MiniOozie ............................. SKIPPED
[INFO] Apache Oozie Distro ................................ SKIPPED
[INFO] Apache Oozie ZooKeeper Security Tests .............. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 44.067 s
[INFO] Finished at: 2017-03-27T17:36:56+08:00
[INFO] Final Memory: 193M/1464M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.3.2:testCompile (default-testCompile) on project oozie-core: Compilation failure: Compilation failure:
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java:[817,47] error: cannot find symbol
[ERROR] variable firstLogEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java:[818,32] error: cannot find symbol
[ERROR] variable firstLogEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java:[819,78] error: cannot find symbol
[ERROR] variable firstLogEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[213,46] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[214,31] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[215,81] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[221,46] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[222,31] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[231,31] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[239,31] error: cannot find symbol
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :oozie-core
- 如果我将 hbase 版本 Ln 85 从
<hbase.version>0.94.27</hbase.version> 更改为 <hbase.version>1.1.1</hbase.version>，我会收到以下错误：
Blockquote
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Oozie Core 4.3.0
[INFO] ------------------------------------------------------------------------
Downloading: http://maven.aliyun.com/nexus/content/repositories/central/org/apache/hbase/hbase/1.1.1/hbase-1.1.1.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Oozie Main .................................. SUCCESS [ 1.008 s]
[INFO] Apache Oozie Hadoop Utils hadoop-2-4.3.0 ........... SUCCESS [ 1.869 s]
[INFO] Apache Oozie Hadoop Distcp hadoop-2-4.3.0 .......... SUCCESS [ 0.094 s]
[INFO] Apache Oozie Hadoop Auth hadoop-2-4.3.0 Test ....... SUCCESS [ 0.287 s]
[INFO] Apache Oozie Hadoop Libs ........................... SUCCESS [ 0.026 s]
[INFO] Apache Oozie Client ................................ SUCCESS [ 14.418 s]
[INFO] Apache Oozie Share Lib Oozie ....................... SUCCESS [ 2.331 s]
[INFO] Apache Oozie Share Lib HCatalog .................... SUCCESS [ 2.478 s]
[INFO] Apache Oozie Share Lib Distcp ...................... SUCCESS [ 0.728 s]
[INFO] Apache Oozie Core .................................. FAILURE [ 0.755 s]
[INFO] Apache Oozie Share Lib Streaming ................... SKIPPED
[INFO] Apache Oozie Share Lib Pig ......................... SKIPPED
[INFO] Apache Oozie Share Lib Hive ........................ SKIPPED
[INFO] Apache Oozie Share Lib Hive 2 ...................... SKIPPED
[INFO] Apache Oozie Share Lib Sqoop ....................... SKIPPED
[INFO] Apache Oozie Examples .............................. SKIPPED
[INFO] Apache Oozie Share Lib Spark ....................... SKIPPED
[INFO] Apache Oozie Share Lib ............................. SKIPPED
[INFO] Apache Oozie Docs .................................. SKIPPED
[INFO] Apache Oozie WebApp ................................ SKIPPED
[INFO] Apache Oozie Tools ................................. SKIPPED
[INFO] Apache Oozie MiniOozie ............................. SKIPPED
[INFO] Apache Oozie Distro ................................ SKIPPED
[INFO] Apache Oozie ZooKeeper Security Tests .............. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 24.879 s
[INFO] Finished at: 2017-03-27T18:01:13+08:00
[INFO] Final Memory: 110M/1420M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project oozie-core: Could not resolve dependencies for project org.apache.oozie:oozie-core:jar:4.3.0: Could not find artifact org.apache.hbase:hbase:jar:1.1.1 in alimaven (http://maven.aliyun.com/nexus/content/repositories/central/) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :oozie-core
我正在使用代理附近的 maven 中央镜像，但这应该不是问题，因为我也尝试了美国镜像，结果是一样的。我已经尽了最大努力查找文档和相关帖子，但到目前为止还没有成功。
请哪位大神赐教
我找不到解决构建问题的合适方法，所以我使用了一个丑陋的解决方法。首先解压 oozie-sharelib-4.3.0.tar.gz，替换导致我的应用程序出现问题的库，然后使用 oozie-setup.sh sharelib create -fs hdfs://<HDFS master node>:9000 -locallib oozie-sharelib-4.3.0.tar.gz 更新共享库。
然而，这也不是正确的方法，因为 oozie 会要求一个看似无穷无尽的 jar 列表。
我终于回到使用shell动作从bash执行sqoop，我不得不解决一个非常奇怪的jar版本冲突问题，这无论如何都超出了本文的范围问题。
我希望丑陋的解决方法可以帮助处于紧急情况的人。
我需要使用 sqoop-1.4.6 参数将配置单元 table 导出到 mysql，其中配置单元 table 有两个分区。但是我发现我的 hadoop 集群上的 oozie 共享库只支持 sqoop-1.4.3。所以我尝试构建和创建符合我们版本规范的共享库。
我修改了oozie pom.xml作为我们的集群主节点的情况，但是如果我做了以下两个更改，构建失败：
- 如果我将配置单元版本 Ln 92 从
<hive.version>1.2.0</hive.version>更改为<hive.version>2.1.0</hive.version>，我会收到以下错误：
Blockquote
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Oozie Main .................................. SUCCESS [ 0.991 s]
[INFO] Apache Oozie Hadoop Utils hadoop-2-4.3.0 ........... SUCCESS [ 1.843 s]
[INFO] Apache Oozie Hadoop Distcp hadoop-2-4.3.0 .......... SUCCESS [ 0.215 s]
[INFO] Apache Oozie Hadoop Auth hadoop-2-4.3.0 Test ....... SUCCESS [ 0.432 s]
[INFO] Apache Oozie Hadoop Libs ........................... SUCCESS [ 0.017 s]
[INFO] Apache Oozie Client ................................ SUCCESS [ 12.492 s]
[INFO] Apache Oozie Share Lib Oozie ....................... SUCCESS [ 1.868 s]
[INFO] Apache Oozie Share Lib HCatalog .................... SUCCESS [ 8.001 s]
[INFO] Apache Oozie Share Lib Distcp ...................... SUCCESS [ 0.733 s]
[INFO] Apache Oozie Core .................................. FAILURE [ 16.672 s]
[INFO] Apache Oozie Share Lib Streaming ................... SKIPPED
[INFO] Apache Oozie Share Lib Pig ......................... SKIPPED
[INFO] Apache Oozie Share Lib Hive ........................ SKIPPED
[INFO] Apache Oozie Share Lib Hive 2 ...................... SKIPPED
[INFO] Apache Oozie Share Lib Sqoop ....................... SKIPPED
[INFO] Apache Oozie Examples .............................. SKIPPED
[INFO] Apache Oozie Share Lib Spark ....................... SKIPPED
[INFO] Apache Oozie Share Lib ............................. SKIPPED
[INFO] Apache Oozie Docs .................................. SKIPPED
[INFO] Apache Oozie WebApp ................................ SKIPPED
[INFO] Apache Oozie Tools ................................. SKIPPED
[INFO] Apache Oozie MiniOozie ............................. SKIPPED
[INFO] Apache Oozie Distro ................................ SKIPPED
[INFO] Apache Oozie ZooKeeper Security Tests .............. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 44.067 s
[INFO] Finished at: 2017-03-27T17:36:56+08:00
[INFO] Final Memory: 193M/1464M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.3.2:testCompile (default-testCompile) on project oozie-core: Compilation failure: Compilation failure:
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java:[817,47] error: cannot find symbol
[ERROR] variable firstLogEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java:[818,32] error: cannot find symbol
[ERROR] variable firstLogEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java:[819,78] error: cannot find symbol
[ERROR] variable firstLogEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[213,46] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[214,31] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[215,81] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[221,46] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[222,31] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[231,31] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[239,31] error: cannot find symbol
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :oozie-core
- 如果我将 hbase 版本 Ln 85 从
<hbase.version>0.94.27</hbase.version>更改为<hbase.version>1.1.1</hbase.version>，我会收到以下错误：
Blockquote
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Oozie Core 4.3.0
[INFO] ------------------------------------------------------------------------
Downloading: http://maven.aliyun.com/nexus/content/repositories/central/org/apache/hbase/hbase/1.1.1/hbase-1.1.1.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Oozie Main .................................. SUCCESS [ 1.008 s]
[INFO] Apache Oozie Hadoop Utils hadoop-2-4.3.0 ........... SUCCESS [ 1.869 s]
[INFO] Apache Oozie Hadoop Distcp hadoop-2-4.3.0 .......... SUCCESS [ 0.094 s]
[INFO] Apache Oozie Hadoop Auth hadoop-2-4.3.0 Test ....... SUCCESS [ 0.287 s]
[INFO] Apache Oozie Hadoop Libs ........................... SUCCESS [ 0.026 s]
[INFO] Apache Oozie Client ................................ SUCCESS [ 14.418 s]
[INFO] Apache Oozie Share Lib Oozie ....................... SUCCESS [ 2.331 s]
[INFO] Apache Oozie Share Lib HCatalog .................... SUCCESS [ 2.478 s]
[INFO] Apache Oozie Share Lib Distcp ...................... SUCCESS [ 0.728 s]
[INFO] Apache Oozie Core .................................. FAILURE [ 0.755 s]
[INFO] Apache Oozie Share Lib Streaming ................... SKIPPED
[INFO] Apache Oozie Share Lib Pig ......................... SKIPPED
[INFO] Apache Oozie Share Lib Hive ........................ SKIPPED
[INFO] Apache Oozie Share Lib Hive 2 ...................... SKIPPED
[INFO] Apache Oozie Share Lib Sqoop ....................... SKIPPED
[INFO] Apache Oozie Examples .............................. SKIPPED
[INFO] Apache Oozie Share Lib Spark ....................... SKIPPED
[INFO] Apache Oozie Share Lib ............................. SKIPPED
[INFO] Apache Oozie Docs .................................. SKIPPED
[INFO] Apache Oozie WebApp ................................ SKIPPED
[INFO] Apache Oozie Tools ................................. SKIPPED
[INFO] Apache Oozie MiniOozie ............................. SKIPPED
[INFO] Apache Oozie Distro ................................ SKIPPED
[INFO] Apache Oozie ZooKeeper Security Tests .............. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 24.879 s
[INFO] Finished at: 2017-03-27T18:01:13+08:00
[INFO] Final Memory: 110M/1420M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project oozie-core: Could not resolve dependencies for project org.apache.oozie:oozie-core:jar:4.3.0: Could not find artifact org.apache.hbase:hbase:jar:1.1.1 in alimaven (http://maven.aliyun.com/nexus/content/repositories/central/) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :oozie-core
我正在使用代理附近的 maven 中央镜像，但这应该不是问题，因为我也尝试了美国镜像，结果是一样的。我已经尽了最大努力查找文档和相关帖子，但到目前为止还没有成功。
请哪位大神赐教
我找不到解决构建问题的合适方法，所以我使用了一个丑陋的解决方法。首先解压 oozie-sharelib-4.3.0.tar.gz，替换导致我的应用程序出现问题的库，然后使用 oozie-setup.sh sharelib create -fs hdfs://<HDFS master node>:9000 -locallib oozie-sharelib-4.3.0.tar.gz 更新共享库。
然而，这也不是正确的方法，因为 oozie 会要求一个看似无穷无尽的 jar 列表。
我终于回到使用shell动作从bash执行sqoop，我不得不解决一个非常奇怪的jar版本冲突问题，这无论如何都超出了本文的范围问题。
我希望丑陋的解决方法可以帮助处于紧急情况的人。