PIG 日期转换发行样本日期为 4/1/2009 9:06:00 AM

Question

bill_details= FOREACH bill_details_raw GENERATE 
              [=11=] AS invD_icode,  AS invD_InvH_icode, 
               AS invD_InvH_No,
              ToDate(invD_InvH_date,'MM/dd/yyyy hh:mm:ss aa') AS dateString,  
               AS invD_pro_icode,
               AS invD_pro_ucode, 
               AS invD_pro_name,  
               AS invD_qty;



java.lang.Exception: org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing [POUserFunc (Name: POUserFunc(org.apache.pig.builtin.ToDate2ARGS)[datetime] - scope-85 Operator Key: scope-85) children: null at []]: java.lang.IllegalArgumentException: Invalid format: " invD_InvH_date"
    at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:489)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:549)
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing [POUserFunc (Name: POUserFunc(org.apache.pig.builtin.ToDate2ARGS)[datetime] - scope-85 Operator Key: scope-85) children: null at []]: java.lang.IllegalArgumentException: Invalid format: " invD_InvH_date"
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:338)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:378)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:298)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:282)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Invalid format: " invD_InvH_date"
    at org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:899)
    at org.apache.pig.builtin.ToDate2ARGS.exec(ToDate2ARGS.java:45)
    at org.apache.pig.builtin.ToDate2ARGS.exec(ToDate2ARGS.java:33)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:330)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNextDateTime(POUserFunc.java:422)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:329)
    ... 14 more

Answer 1

我能够毫无问题地将给定的字符串转换为日期。这是有效的代码。

date.txt

4/1/2009 9:06:00 AM

代码

file_date = load 'date.txt';
dates_only = foreach file_data generate ToDate([=11=], 'MM/dd/yyyy hh:mm:ss aa');

输出

dump file_date;
(4/1/2009 9:06:00 AM)
dump dates_only
(2009-04-01T09:06:00.000Z)

可能的解决方案：

我认为问题是，您正在使用列名，现在我不确定它是否是架构的一部分，但似乎不是。

你能试试这个吗？

bill_details= FOREACH bill_details_raw GENERATE 
              [=13=] AS invD_icode, 
               AS invD_InvH_icode, 
               AS invD_InvH_No,
              ToDate(,'MM/dd/yyyy hh:mm:ss aa') AS dateString,  
               AS invD_pro_icode,
               AS invD_pro_ucode, 
               AS invD_pro_name,  
               AS invD_qty;

Answer 2

bill_details= FOREACH bill_details_raw GENERATE [=10=] AS invD_icode,
              AS invD_InvH_icode, 
              AS invD_InvH_No, 
            ToDate(,' M/dd/yyyy hh:mm:ss aa') AS dateString,
             AS invD_pro_icode, 
             AS invD_pro_ucode, 
             AS invD_pro_name, 
             AS invD_qty;

PIG 日期转换发行样本日期为 4/1/2009 9:06:00 AM

PIG date conversion issue sample date is 4/1/2009 9:06:00 AM

hadoop

apache-pig