Oozie 自定义 EL 函数 oozie for coordinator
Oozie Custom EL function oozie for coordinator
我想创建一个我将在 oozie 协调器中使用的自定义 el 函数。我的自定义 el 函数工作正常,但是当我将已经定义的 oozie el 函数作为参数传递给我的 el 函数时,它会抛出异常。
coordinator.xml
${coord:dateToEpoch(coord:nominalTime(), "yyyy-MM-dd'T'hh:mmZ") 见下例
<datasets>
<dataset name="input1" frequency="${inputDataSetFrequence}" initial-instance="${initialInstance}"
timezone="${timezone}">
<uri-template>${inputBasePath}/${useCaseName}/bintime=${coord:dateToEpoch(coord:nominalTime(), "yyyy-MM-dd'T'hh:mmZ")}
</uri-template>
<done-flag></done-flag>
</dataset>
</datasets>
<input-events>
<data-in name="coordInput1" dataset="input1">
<instance>${coord:current(0)}</instance>
</data-in>
</input-events>
我用来测试这个的配置
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-nofuncs</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
常量参数有效但动态参数无效
工作通话
${coord:dateToEpoch("2009-01-01T08:00UTC", "yyyy-MM-dd'T'hh:mmZ")
异常调用
${coord:dateToEpoch(coord:nominalTime(), "yyyy-MM-dd'T'hh:mmZ")
尝试使用所有这些属性
<property>
<name>oozie.service.ELService.ext.functions.job-submit</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.workflow</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.wf-sla-submit</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-freq</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-data</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-instances</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-sla-create</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-sla-submit</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-action-create</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-action-create-inst</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-action-start</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-wait-timeout</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.bundle-submit</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-initial-instance</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
和
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-nofuncs</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
异常
Caused by: java.lang.Exception: Unable to evaluate :${inputBasePath}/${useCaseName}/bintime=${coord:dateToEpoch(coord:nominalTime(), "yyyy-MM-dd'T'hh:mmZ")}:
at org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:743)
at org.apache.oozie.command.coord.CoordSubmitXCommand.resolveTagContents(CoordSubmitXCommand.java:1002)
... 37 more
Caused by: javax.servlet.jsp.el.ELException: No function is mapped to the name "coord:nominalTime"
at org.apache.commons.el.Logger.logError(Logger.java:481)
at org.apache.commons.el.Logger.logError(Logger.java:498)
at org.apache.commons.el.Logger.logError(Logger.java:525)
at org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:150)
at org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:163)
at org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114)
at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274)
at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:204)
at org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:734)
并非所有 Oozie
EL 构造都在 uri-template
中进行评估。请参阅 Synchronous Datasets 部分了解更多详细信息。以下是 uri-template
:
的摘录
uri-template: The URI template that identifies the dataset and can be
resolved into concrete URIs to identify a particular dataset instance.
The URI template is constructed using:
constants: See the allowable EL Time Constants below. Ex: ${YEAR}/${MONTH}.
variables: Variables must be resolved at the time a coordinator job is submitted to the coordinator engine. They are normally provided a job parameters (configuration properties). Ex: ${market}/${language}
可以在同步数据集 URI 模板中使用以下 EL 常量:
- 年
- 月份
- 天
- 小时
- 分钟
问题与您的自定义 EL 函数实现无关。所以即使你用了following,也不行:
<uri-template>${inputBasePath}/${useCaseName}/bintime=${coord:nominalTime()}</uri-template>
我想创建一个我将在 oozie 协调器中使用的自定义 el 函数。我的自定义 el 函数工作正常,但是当我将已经定义的 oozie el 函数作为参数传递给我的 el 函数时,它会抛出异常。
coordinator.xml
${coord:dateToEpoch(coord:nominalTime(), "yyyy-MM-dd'T'hh:mmZ") 见下例
<datasets>
<dataset name="input1" frequency="${inputDataSetFrequence}" initial-instance="${initialInstance}"
timezone="${timezone}">
<uri-template>${inputBasePath}/${useCaseName}/bintime=${coord:dateToEpoch(coord:nominalTime(), "yyyy-MM-dd'T'hh:mmZ")}
</uri-template>
<done-flag></done-flag>
</dataset>
</datasets>
<input-events>
<data-in name="coordInput1" dataset="input1">
<instance>${coord:current(0)}</instance>
</data-in>
</input-events>
我用来测试这个的配置
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-nofuncs</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
常量参数有效但动态参数无效
工作通话
${coord:dateToEpoch("2009-01-01T08:00UTC", "yyyy-MM-dd'T'hh:mmZ")
异常调用
${coord:dateToEpoch(coord:nominalTime(), "yyyy-MM-dd'T'hh:mmZ")
尝试使用所有这些属性
<property>
<name>oozie.service.ELService.ext.functions.job-submit</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.workflow</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.wf-sla-submit</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-freq</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-data</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-instances</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-sla-create</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-sla-submit</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-action-create</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-action-create-inst</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-action-start</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-wait-timeout</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.bundle-submit</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-initial-instance</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
和
<property>
<name>oozie.service.ELService.ext.functions.coord-job-submit-nofuncs</name>
<value>coord:dateToEpoch=com.mobileum.oozie.MobileumELFunctions#dateToEpoch</value>
</property>
异常
Caused by: java.lang.Exception: Unable to evaluate :${inputBasePath}/${useCaseName}/bintime=${coord:dateToEpoch(coord:nominalTime(), "yyyy-MM-dd'T'hh:mmZ")}:
at org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:743)
at org.apache.oozie.command.coord.CoordSubmitXCommand.resolveTagContents(CoordSubmitXCommand.java:1002)
... 37 more
Caused by: javax.servlet.jsp.el.ELException: No function is mapped to the name "coord:nominalTime"
at org.apache.commons.el.Logger.logError(Logger.java:481)
at org.apache.commons.el.Logger.logError(Logger.java:498)
at org.apache.commons.el.Logger.logError(Logger.java:525)
at org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:150)
at org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:163)
at org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114)
at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274)
at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:204)
at org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:734)
并非所有 Oozie
EL 构造都在 uri-template
中进行评估。请参阅 Synchronous Datasets 部分了解更多详细信息。以下是 uri-template
:
uri-template: The URI template that identifies the dataset and can be resolved into concrete URIs to identify a particular dataset instance. The URI template is constructed using:
constants: See the allowable EL Time Constants below. Ex: ${YEAR}/${MONTH}.
variables: Variables must be resolved at the time a coordinator job is submitted to the coordinator engine. They are normally provided a job parameters (configuration properties). Ex: ${market}/${language}
可以在同步数据集 URI 模板中使用以下 EL 常量:
- 年
- 月份
- 天
- 小时
- 分钟
问题与您的自定义 EL 函数实现无关。所以即使你用了following,也不行:
<uri-template>${inputBasePath}/${useCaseName}/bintime=${coord:nominalTime()}</uri-template>