M/R job submission failing with error: Could not find Yarn tags property > (mapreduce.job.tags)
M/R job submission failing with error: Could not find Yarn tags property > (mapreduce.job.tags)
我在 运行 map/reduce 工作时遇到以下异常。我们通过 oozie 提交 map/reduce 个职位。
Failing Oozie Launcher, Main class
[org.apache.oozie.action.hadoop.JavaMain], main() threw exception,
Could not find Yarn tags property (mapreduce.job.tags)
java.lang.RuntimeException: Could not find Yarn tags property
(mapreduce.job.tags) at
org.apache.oozie.action.hadoop.LauncherMainHadoopUtils.getChildYarnJobs(LauncherMainHadoopUtils.java:53)
at
org.apache.oozie.action.hadoop.LauncherMainHadoopUtils.killChildYarnJobs(LauncherMainHadoopUtils.java:88)
at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:46) at
org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:46)
at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:38) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606) at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:228)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:378)
at
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:296)
at
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access0(LocalContainerLauncher.java:181)
at
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.run(LocalContainerLauncher.java:224)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745
我进行了 google 搜索，发现了以下 SO post： Hadoop MapReduce job starts but can not find Map class? 但是 post 中提到的解决方案对我不起作用，我不能在日志文件中查看任何与文件权限相关的错误。
我们正在使用 Cloudera 发行版。
您需要升级 Oozie 共享库。按照说明 in Cloudera's documentation。即：
sudo oozie-setup sharelib create -fs FS_URI -locallib /usr/lib/oozie/oozie-sharelib-yarn
之后不要忘记重新启动 Oozie。这帮助我们解决了 CDH 5.5 升级后的这个特殊问题。
我在 运行 map/reduce 工作时遇到以下异常。我们通过 oozie 提交 map/reduce 个职位。
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.JavaMain], main() threw exception, Could not find Yarn tags property (mapreduce.job.tags) java.lang.RuntimeException: Could not find Yarn tags property (mapreduce.job.tags) at org.apache.oozie.action.hadoop.LauncherMainHadoopUtils.getChildYarnJobs(LauncherMainHadoopUtils.java:53) at org.apache.oozie.action.hadoop.LauncherMainHadoopUtils.killChildYarnJobs(LauncherMainHadoopUtils.java:88) at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:46) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:46) at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:38) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:228) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:378) at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:296) at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access0(LocalContainerLauncher.java:181) at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.run(LocalContainerLauncher.java:224) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745
我进行了 google 搜索，发现了以下 SO post： Hadoop MapReduce job starts but can not find Map class? 但是 post 中提到的解决方案对我不起作用，我不能在日志文件中查看任何与文件权限相关的错误。
我们正在使用 Cloudera 发行版。
您需要升级 Oozie 共享库。按照说明 in Cloudera's documentation。即：
sudo oozie-setup sharelib create -fs FS_URI -locallib /usr/lib/oozie/oozie-sharelib-yarn
之后不要忘记重新启动 Oozie。这帮助我们解决了 CDH 5.5 升级后的这个特殊问题。