Hadoop YarnRuntimeException

Question

我运行我在 Hadoop 2.6 上的 mapreduce 作业出现了这个错误。请告诉我如何解决这个问题。

诊断：

Application application_1425522415042_0001 failed 2 times due to AM Container for appattempt_1425522415042_0001_000002 exited with exitCode: 1 
For more detailed output, check application tracking page:http://xxx:8088/proxy/application_1425522415042_0001/Then, click on links to logs of each attempt. 
Diagnostics: Exception from container-launch. 
Container id: container_1425522415042_0001_02_000001 
Exit code: 1 
Stack trace: ExitCodeException exitCode=1: 

at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) 
at org.apache.hadoop.util.Shell.run(Shell.java:455) 
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) 
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) 
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) 
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) 
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)

容器以非零退出代码 1 退出此尝试失败。申请失败。

节点日志

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/exceptions/YarnRuntimeException 
     at java.lang.Class.getDeclaredMethods0(Native Method) 
     at java.lang.Class.privateGetDeclaredMethods(Class.java:2693) 
     at java.lang.Class.privateGetMethodRecursive(Class.java:3040) 
     at java.lang.Class.getMethod0(Class.java:3010) 
     at java.lang.Class.getMethod(Class.java:1776) 
     at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) 
     at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) 
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.exceptions.YarnRuntimeException 
     at java.net.URLClassLoader.run(URLClassLoader.java:372) 
     at java.net.URLClassLoader.run(URLClassLoader.java:361) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at java.net.URLClassLoader.findClass(URLClassLoader.java:360) 
     at java.lang.ClassLoader.loadClass(ClassLoader.java:424) 
     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) 
     at java.lang.ClassLoader.loadClass(ClassLoader.java:357) 
     ... 7 more

纱-site.xml

<configuration>

<!-- Site specific YARN configuration properties -->
<property>
    <name>yarn.resourcemanager.hostname</name>
    <value>master.lab.buptsse.cn</value>
</property>
<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>
<property>
    <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>  
       <name>yarn.application.classpath</name>  
       <value>
            $HADOOP_CONF_DIR,  
            $HADOOP_COMMON_HOME/share/hadoop/common/*,  
            $HADOOP_COMMON_HOME/share/hadoop/common/lib/*,  
            $HADOOP_HDFS_HOME/share/hadoop/hdfs/*,  
            $HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*,  
            $YARN_HOME/share/hadoop/yarn/*
            $YARN_HOME/share/hadoop/yarn/lib/*
            $YARN_HOME/share/hadoop/mapreduce/*,  
            $YARN_HOME/share/hadoop/mapreduce/lib/*  
       </value>  
</property>
</configuration>

mapred-site.xml

<configuration> 
<property> 
     <name>mapreduce.framework.name</name> 
     <value>yarn</value> 
</property> 
</configuration>

.bashrc

export HADOOP_HOME=/home/hadoop/hadoop-2.6.0 
export HADOOP_MAPRED_HOME=$HADOOP_HOME 
export HADOOP_COMMON_HOME=$HADOOP_HOME 
export HADOOP_HDFS_HOME=$HADOOP_HOME 
export YARN_HOME=$HADOOP_HOME 
export HADOOP_YARN_HOME=$HADOOP_HOME 
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop 
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop 
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native 
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

Answer 1

今天我运行遇到同样的问题，发现hadoop-yarn-api-2.6.0.jar文件中包含YarnRuntimeException。确保它在你的 class 路径中，你应该没问题。

Answer 2

不设置

    conf.setAppName("name");
    conf.setMaster("local");

对于 SparkConf

Hadoop YarnRuntimeException

Hadoop YarnRuntimeException

java

hadoop2