将参数 "args" 从 main class 传递到 Map class

Question

例如：jar class arg1 arg2 arg3

arg 1 用于输入格式，arg 2 用于输出格式，如下所示：

public static void main(String[] args)
{
FileInputFormat.addInputPath(conf, new Path(args[0]));
FileOutputFormat.setOutputPath(conf, new Path(args[1]));
.
.
.
.
}

我需要将 arg3 "args[2]" 发送到地图 class .....

public class JoinMultiMap extends MapReduceBase
  implements Mapper<LongWritable, Text, Text, Text> 
{
i need arg3 her
}

Answer 1

您可以使用 Configuration class 来设置和获取自定义配置属性，如下所示：

在您的驱动程序代码中：

import org.apache.hadoop.conf.Configuration;
// other imports ignored for brevity
// ...

public class YourDriver extends Configured implements Tool {
  public int run(String[] args) {
    Configuration conf = getConf();
    conf.set("yourcustom.property", args[2]);
    // other driver code ignored
    // ...
  }

  public static void main(String[] args)  {
    int res = ToolRunner.run(new Configuration(), new YourDriver(), args);
    System.exit(res);
  }
}

来自映射器代码：

public class YourMapper extends Mapper<...> {
  private String yourArgument;

  @Override
    protected void setup(Context context) {
        Configuration c = context.getConfiguration();
        yourArgument = c.get("yourcustom.property");
    }

  @Override
  protected void map(...) {
    // use your argument
  }
}

将参数 "args" 从 main class 传递到 Map class

passing argument "args" from main class to Map class

hadoop

mapreduce

command-line-arguments