Pig 在尝试 Edureka 的教程时出错
Pig Error trying Edureka's Tutorial
我一直在尝试 运行 hadoop 和 PIG 等其他组件。
我正在尝试本教程:
https://www.edureka.co/blog/pig-programming-create-your-first-apache-pig-script/
一切正常,但是当我在第 2 步 运行 脚本时,它抛出此错误:
2018-01-09 13:47:20,682 [JobControl] INFO org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob - PigLatin:output.pig got an error while submitting
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: hdfs://localhost:9000/carlos/information.txt
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:279)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
at org.apache.hadoop.mapreduce.Job.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:194)
at java.lang.Thread.run(Thread.java:748)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.run(MapReduceLauncher.java:276)
在第 1 步之前,您需要此命令
hadoop fs –copyFromLocal /home/carlos/information.txt /carlos
但是,如果该目录尚不存在,这会将您的文件复制到 HDFS 上名为 /carlos
的 文件。
如果你想让/carlos
成为一个目录,你需要把文件删掉,然后做成
hadoop fs -rm /carlos
hadoop fs -mkdir /carlos
此外,将文件复制到目录时通常应使用尾部斜杠,例如
hadoop fs –copyFromLocal /home/carlos/information.txt /carlos/
您也可以将 Pig 代码作为文件加载 /carlos
。即使它是一个目录,这仍然可以读取其中的所有文件
我一直在尝试 运行 hadoop 和 PIG 等其他组件。
我正在尝试本教程: https://www.edureka.co/blog/pig-programming-create-your-first-apache-pig-script/
一切正常,但是当我在第 2 步 运行 脚本时,它抛出此错误:
2018-01-09 13:47:20,682 [JobControl] INFO org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob - PigLatin:output.pig got an error while submitting
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: hdfs://localhost:9000/carlos/information.txt
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:279)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
at org.apache.hadoop.mapreduce.Job.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:194)
at java.lang.Thread.run(Thread.java:748)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.run(MapReduceLauncher.java:276)
在第 1 步之前,您需要此命令
hadoop fs –copyFromLocal /home/carlos/information.txt /carlos
但是,如果该目录尚不存在,这会将您的文件复制到 HDFS 上名为 /carlos
的 文件。
如果你想让/carlos
成为一个目录,你需要把文件删掉,然后做成
hadoop fs -rm /carlos
hadoop fs -mkdir /carlos
此外,将文件复制到目录时通常应使用尾部斜杠,例如
hadoop fs –copyFromLocal /home/carlos/information.txt /carlos/
您也可以将 Pig 代码作为文件加载 /carlos
。即使它是一个目录,这仍然可以读取其中的所有文件