将参数传递给 pig udf 无法使用参数实例化

Question

我正在尝试编写可以接受参数的 Pig UDF。

我看了一些博客，了解到可以通过添加参数化构造函数来完成。

所以我的 UDF 代码是：

import java.io.IOException;
import java.util.List;

import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
import org.apache.commons.lang.StringUtils;
public class LeftPad extends EvalFunc<String> {

    String size;
    String padChar;
    public LeftPad(String size, String padChar) {
        // TODO Auto-generated constructor stub
        this.size = size;
        this.padChar = padChar;
    }

    @Override
    public String exec(Tuple input) throws IOException {
        try {
            if (input == null || input.size() == 0)
                return null;
            String str = (String) input.get(0);         
            return StringUtils.leftPad(str, Integer.valueOf(size), padChar).toString();

        } catch (Exception exception) {
            exception.printStackTrace();
        }
        return null;
    }
}

还尝试将构造函数定义为：

public LeftPad(int size, Char padChar)

但是阅读某处它只支持String并更改为上面。

grunt 控制台上使用的脚本行：

REGISTER pig_udf4.jar
DEFINE l_pad pig_udf4.LeftPad('12','x');
data = Load myData.csv USING PigStorage(,) as (....);
C = FOREACH data GENERATE l_pad('12','x');
DUMP C;

尝试将第 4 行更改为

C = FOREACH data GENERATE l_pad();

但仍然面临同样的问题。

日志：

Pig Stack Trace
---------------
ERROR 1200: could not instantiate 'LeftPad' with arguments '[12, x]'

Failed to parse: could not instantiate 'LeftPad' with arguments '[12, x]'
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:201)
        at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1791)
        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1764)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:707)
        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1075)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:505)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:231)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66)
        at org.apache.pig.Main.run(Main.java:564)
        at org.apache.pig.Main.main(Main.java:176)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: could not instantiate 'LeftPad' with arguments '[12, x]'
        at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:759)
        at org.apache.pig.newplan.logical.expression.UserFuncExpression.getFieldSchema(UserFuncExpression.java:222)
        at org.apache.pig.newplan.logical.optimizer.FieldSchemaResetter.execute(SchemaResetter.java:264)
        at org.apache.pig.newplan.logical.expression.AllSameExpressionVisitor.visit(AllSameExpressionVisitor.java:143)
        at org.apache.pig.newplan.logical.expression.UserFuncExpression.accept(UserFuncExpression.java:113)
        at org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70)
        at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
        at org.apache.pig.newplan.logical.optimizer.SchemaResetter.visitAll(SchemaResetter.java:67)
        at org.apache.pig.newplan.logical.optimizer.SchemaResetter.visit(SchemaResetter.java:122)
        at org.apache.pig.newplan.logical.relational.LOGenerate.accept(LOGenerate.java:245)
        at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
        at org.apache.pig.newplan.logical.optimizer.SchemaResetter.visit(SchemaResetter.java:114)
        at org.apache.pig.parser.LogicalPlanBuilder.buildForeachOp(LogicalPlanBuilder.java:1055)
        at org.apache.pig.parser.LogicalPlanGenerator.foreach_clause(LogicalPlanGenerator.java:15896)
        at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1933)
        at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
        at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
        at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
        ... 16 more
Caused by: java.lang.NoSuchMethodException: LeftPad.<init>([Ljava.lang.String;)
        at java.lang.Class.getConstructor0(Class.java:3082)
        at java.lang.Class.getConstructor(Class.java:1825)
        at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:746)
        ... 34 more
================================================================================

Answer 1

NoSuchMethodException: LeftPad.([Ljava.lang.String;)

你给出了一个字符串数组，而不是两个字符串。

再添加一个构造器

public LeftPad(String... args) {
    this(args[0], args[1]);
}

除此之外，我怀疑你是否需要 DEFINE。

您的参数被传递到输入元组中，因此您可以从中获取大小和字符

将参数传递给 pig udf 无法使用参数实例化

passing parameters to pig udf could not instantiate with arguments

java

hadoop

user-defined-functions

apache-pig

udf