Pig Java UDF：从元组生成包

Question

我希望有人可以帮助我创建一个 java UDF，它将把这个输入分布在三个文本文件中：

Montreal, 5 3 10 9 8
Toronto, 7 2 2 3 4 4
Edmonton, 3 3 1 1 7
Montreal, 2 2 9

和return以下输出包：

{(Montreal,5),(Montreal,3),(Montreal,10),(Montreal,9),(Montreal,8),(Montreal,2),(Montreal,2),(Montreal,9)}
{(Toronto,7),(Toronto,2),(Toronto,2),(Toronto,3),(Toronto,4),(Toronto,4)}

我是 java 的新手，非常感谢您提供的任何帮助。谢谢。

Answer 1

如果您使用的是 pig 0.14 或更高版本支持 STRSPLITTOBAG，那么

A = load 'test.input' using PigStorage(',') as (place:chararray, numbers:chararray);
B = FOREACH A GENERATE place, FLATTEN(STRSPLITTOBAG(numbers)) as number;
C = FOREACH B GENERATE place, (chararray) number;
D = GROUP C by place;
E = FOREACH D generate C; -- dropping group field
dump E;

输出

({(Toronto,2),(Toronto,2),(Toronto,7),(Toronto,4),(Toronto,4),(Toronto,3)})
({(Edmonton,7),(Edmonton,1),(Edmonton,1),(Edmonton,3),(Edmonton,3)})
({(Montreal,9),(Montreal,2),(Montreal,2),(Montreal,8),(Montreal,9),(Montreal,10),(Montreal,3),(Montreal,5)})

Pig Java UDF：从元组生成包

Pig Java UDF: Generating a bag from a tuple

apache-pig

user-defined-functions