Oozie java-动作不包含核心-site.xml

Question

当运行在新安装的 Hadoop HDP 2.2.2.4 上执行 Oozie java 操作时，例如尝试访问 hdfs，它会访问错误的文件系统： java.lang.IllegalArgumentException: Wrong FS: hdfs:/tmp/text.txt, expected: file:///

可以通过在 Oozie 操作中包含核心-site.xml 来修复它：

<file>hdfs:/path-to-core-site.xml-on-hdfs</file>

但是原因是什么，正确的解决方法是什么？

Answer 1

核心-site.xml不包含在java-动作的class-路径中的原因是因为属性mapreduce.application.classpath指向错误的目录：

<snip>/etc/hadoop/conf/secure

应该指向

<snip>/etc/hadoop/conf

即，完整的属性应该类似于 mapred-site.xml:

<property>
  <name>mapreduce.application.classpath</name>
  <value>$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:/usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar:/etc/hadoop/conf</value>
</property>

Answer 2

这些文件包含在 hadoop 类路径中，据我所知，自 HDP 2.2 以来，您需要添加

// loading action conf prepared by Oozie
Configuration actionConf = new Configuration(false);
actionConf.addResource(new Path("file:///", System.getProperty("oozie.action.conf.xml")));

使用*-site.xml，可以在oozie文档中获取详细信息 https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a3.2.7_Java_Action

Oozie java-动作不包含核心-site.xml

Oozie java-action does not include core-site.xml

hadoop

hdfs

oozie