无法运行我在 mapreduce 中的第一个 wordcount 示例

Question

谁能帮我解决这个问题？
我是hadoop的初学者。

我运行我的程序遵循的步骤：

export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export PATH=${JAVA_HOME}/bin:${PATH}
export HADOOP_CLASSPATH=${JAVA_HOME}/lib/tools.jar
bin/hadoop com.sun.tools.javac.Main WordCount.java 
sudo jar cf wc.jar WordCount*.class
bin/hadoop jar wc.jar WordCount myfileName outputFile

这是我尝试运行我的程序时得到的结果

abbas@abbas:~/hadoop-2.7.3$ export JAVA_HOME=/usr/lib/jvm/java-8-oracle
abbas@abbas:~/hadoop-2.7.3$ export PATH=${JAVA_HOME}/bin:${PATH}
abbas@abbas:~/hadoop-2.7.3$ export HADOOP_CLASSPATH=${JAVA_HOME}/lib/tools.jar
abbas@abbas:~/hadoop-2.7.3$ bin/hadoop com.sun.tools.javac.Main WordCount.java 
abbas@abbas:~/hadoop-2.7.3$ sudo jar cf wc.jar WordCount*.class
abbas@abbas:~/hadoop-2.7.3$ bin/hadoop jar wc.jar WordCount tempfile output8

17/06/09 02:20:24 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/06/09 02:20:25 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
17/06/09 02:20:26 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/abbas/.staging/job_1496953867826_0001

Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://localhost:9000/user/abbas/tempfile
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
    at org.apache.hadoop.mapreduce.Job.run(Job.java:1290)
    at org.apache.hadoop.mapreduce.Job.run(Job.java:1287)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
    at WordCount.main(WordCount.java:59)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Answer 1

不言自明。

Input path does not exist: hdfs://localhost:9000/user/abbas/tempfile

你有hdfs用户目录吗？

hdfs dfs -ls /user | grep $(whoami)

你有没有把任何名为 tempfile 的文件放在那里？

hdfs dfs -mkdir -p "/user/$(whoami)"
hdfs dfs -copyFromLocal /path/to/tempfile "/user/$(whoami)"

值得一提的是the example assumes you have files in HDFS already

无法 运行 我在 mapreduce 中的第一个 wordcount 示例

Unable to run my first wordcount example in mapreduce

hadoop

mapreduce

无法运行我在 mapreduce 中的第一个 wordcount 示例