Oozie 失败 - 运行 CDH 5.4.2 上的 MRv1
Oozie Fails - Running MRv1 on CDH 5.4.2
我正在尝试 运行 应该执行 MRv1 hadoop 作业的 oozie 工作流。
从 Cloudera QuickStart VM 5.4.2-0 开始。将其配置为使用 MRv1(附加在底部)。
但工作流失败:
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.MapReduceMain], main() threw exception, org/apache/hadoop/yarn/exceptions/YarnException
java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/exceptions/YarnException
at org.apache.oozie.action.hadoop.MapReduceMain.run(MapReduceMain.java:58)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:46)
at org.apache.oozie.action.hadoop.MapReduceMain.main(MapReduceMain.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:228)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.Child.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.exceptions.YarnException
at java.net.URLClassLoader.run(URLClassLoader.java:366)
at java.net.URLClassLoader.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
奇怪的是,当我查看类路径时,它正在从正确的位置加载 jar 文件;即/usr/lib/hadoop-0.20-mapreduce
-----配置----
配置 OOzie 以使用 MapReduce 而不是纱线:
- 通过更改更改了 CDH 中 Oozie 的配置:MapReduce 服务从
YARN
到 MapReduce
通过设置 oozie-site.xml 阀值更改了 Oozie 的配置:
<property>
<name>oozie.service.HadoopAccessorService.jobTracker.whitelist</name>
<value>quickstart.cloudera:8021</value>
</property>
使用备选方案更改了 tomcat 配置:
sudo alternatives --config oozie-tomcat-deployment -- chose tomcat-conf.http.mr1
配置 Hadoop 以使用 MapReduce:
sudo alternatives --config hadoop-conf -- chose conf.cloudera.mapreduce
我忘记更新操作以使用 MR1:
$ sudo -u oozie hadoop fs -rmr /user/oozie/share
sudo oozie-setup sharelib create -fs <FS_URI> -locallib /usr/lib/oozie/oozie-sharelib-mr1
我正在尝试 运行 应该执行 MRv1 hadoop 作业的 oozie 工作流。
从 Cloudera QuickStart VM 5.4.2-0 开始。将其配置为使用 MRv1(附加在底部)。
但工作流失败:
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.MapReduceMain], main() threw exception, org/apache/hadoop/yarn/exceptions/YarnException
java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/exceptions/YarnException
at org.apache.oozie.action.hadoop.MapReduceMain.run(MapReduceMain.java:58)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:46)
at org.apache.oozie.action.hadoop.MapReduceMain.main(MapReduceMain.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:228)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.Child.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.exceptions.YarnException
at java.net.URLClassLoader.run(URLClassLoader.java:366)
at java.net.URLClassLoader.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
奇怪的是,当我查看类路径时,它正在从正确的位置加载 jar 文件;即/usr/lib/hadoop-0.20-mapreduce
-----配置----
配置 OOzie 以使用 MapReduce 而不是纱线:
- 通过更改更改了 CDH 中 Oozie 的配置:MapReduce 服务从
YARN
到MapReduce
通过设置 oozie-site.xml 阀值更改了 Oozie 的配置:
<property> <name>oozie.service.HadoopAccessorService.jobTracker.whitelist</name> <value>quickstart.cloudera:8021</value> </property>
使用备选方案更改了 tomcat 配置:
sudo alternatives --config oozie-tomcat-deployment -- chose tomcat-conf.http.mr1
- 通过更改更改了 CDH 中 Oozie 的配置:MapReduce 服务从
配置 Hadoop 以使用 MapReduce:
sudo alternatives --config hadoop-conf -- chose conf.cloudera.mapreduce
我忘记更新操作以使用 MR1:
$ sudo -u oozie hadoop fs -rmr /user/oozie/share
sudo oozie-setup sharelib create -fs <FS_URI> -locallib /usr/lib/oozie/oozie-sharelib-mr1