猪拉丁无法倾倒

Question

我无法使用别名 'TMP' 的转储功能。加载和转储文件时我没有遇到任何问题。我对两个表以及别名 'TMP' 使用 describe 'function' 没有问题。我还手动将路径设置为该文件夹，因此应该没有任何问题。这些文件也在 HDFS 服务器上。

orderdetails = load 'order_details.tbl' using PigStorage('|') as
(ORDER_ID:int,PRODUCT_ID:int,CUSTOMER_ID:int,SALESPERSON_ID:int,UNIT_PRICE:float,QUANTITY:int,DISCOUNT:float                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     );
dump orderdetails;
describe orderdetails;

salesperson = load 'salesperson.tbl' using PigStorage('|') as
(EMPLOYEE_ID:int,LASTNAME:chararray,FIRSTNAME:chararray,TITLE:chararray,BIRTHDATE:chararray,HIREDATE:chararray,NOTES:chararray                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        );
dump salesperson;
describe salesperson;

TMP = join salesperson by EMPLOYEE_ID, orderdetails by SALESPERSON_ID;
dump TMP;
describe TMP;

错误：

Failed!

Failed Jobs:
JobId   Alias   Feature Message Outputs
job_1622041116748_0009  TMP,orderdetails,salesperson    HASH_JOIN   Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: hdfs://localhost:8020/user/bigdata/A3/order_details.tbl
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:279)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
    at org.apache.hadoop.mapreduce.Job.run(Job.java:1290)
    at org.apache.hadoop.mapreduce.Job.run(Job.java:1287)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
    at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
    at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:194)
    at java.lang.Thread.run(Thread.java:748)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.run(MapReduceLauncher.java:276)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://localhost:8020/user/bigdata/A3/order_details.tbl
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:265)
    ... 18 more
    hdfs://localhost:8020/tmp/temp441281403/tmp921242953,

Input(s):
Failed to read data from "hdfs://localhost:8020/user/bigdata/A3/salesperson.tbl"
Failed to read data from "hdfs://localhost:8020/user/bigdata/A3/order_details.tbl"

Output(s):
Failed to produce result in "hdfs://localhost:8020/tmp/temp441281403/tmp921242953"

Answer 1

根据错误，提供的路径不存在。请提供正确的路径。

Relation_name = LOAD 'Input file path' USING function as schema;

它可能看起来像

student = LOAD 'hdfs://localhost:9000/pig_data/student_data.txt' USING PigStorage(',') as ( id:int, firstname:chararray,lastname:chararray, phone:chararray, city:chararray );

猪拉丁无法倾倒

Pig Latin unable to dump

ubuntu

hadoop

apache-pig