Oozie - 无法识别 nameNode "maprfs:///"
Oozie - Not Recognizing "maprfs:///" for nameNode
我在自定义 MapR 沙箱中 运行ning Oozie。下面是我的 job.properties 文件信息：
nameNode=maprfs:///
jobTracker=maprfs:///
queueName=default
examplesRoot=examples
oozie.wf.application.path=/user/user01/lab2/exercise4/zayd_hammoudeh_lab2_exercise4/workflow.xml
作为参考，下面是我的 workflow.xml 文件：
<action name="stdev-calc">
<map-reduce>
<job-tracker>${jobTracker}</job-tracker>
<name-node>"${nameNode}"</name-node>
<prepare>
<delete path="/user/user01/lab2/exercise4/OUT"/>
</prepare>
<configuration>
<property>
<name>mapred.mapper.class</name>
<value>Stdev.StdevMapper</value>
</property>
<property>
<name>mapred.reducer.class</name>
<value>Stdev.StdevReducer</value>
</property>
<property>
<name>mapred.input.dir</name>
<value>/user/user01/lab2/exercise4/DATA</value>
</property>
<property>
<name>mapred.output.dir</name>
<value>/user/user01/lab2/exercise4/OUT</value>
</property>
</configuration>
</map-reduce>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Exercise 4 failed with error message: [${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
当我尝试 运行 我的工作时，我在我的工作信息中得到以下信息：
Job ID : 0000004-151018210741238-oozie-mapr-W
------------------------------------------------------------------------------------------------------------------------------------
Workflow Name : aggregator-wf
App Path : /user/user01/lab2/exercise4/zayd_hammoudeh_lab2_exercise4/workflow.xml
Status : KILLED
Run : 0
User : user01
Group : -
Created : 2015-10-19 04:42 GMT
Started : 2015-10-19 04:42 GMT
Last Modified : 2015-10-19 04:42 GMT
Ended : 2015-10-19 04:42 GMT
CoordAction ID: -
Actions
------------------------------------------------------------------------------------------------------------------------------------
ID Status Ext ID Ext Status Err Code
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@:start: OK - OK -
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@stdev-calc ERROR - ERROR IllegalArgumentException
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@fail OK - OK E0729
------------------------------------------------------------------------------------------------------------------------------------
我的日志显示错误为：“[IllegalArgumentException：索引 0 处方案名称中的非法字符："maprfs:///"]”供参考，下面是我的日志：
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/mapr/oozie/oozie-4.1.0/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/mapr/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
2015-10-18 21:42:06,368 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] Start action [0000004-151018210741238-oozie-mapr-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,369 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] [***0000004-151018210741238-oozie-mapr-W@:start:***]Action status=DONE
2015-10-18 21:42:06,372 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] [***0000004-151018210741238-oozie-mapr-W@:start:***]Action updated in DB!
2015-10-18 21:42:06,413 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Start action [0000004-151018210741238-oozie-mapr-W@stdev-calc] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,468 WARN ActionStartXCommand:544 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Error starting action [stdev-calc]. ErrorType [ERROR], ErrorCode [IllegalArgumentException], Message [IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"]
org.apache.oozie.action.ActionExecutorException: IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"
at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:401)
at org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:532)
at org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:666)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:281)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"
at java.net.URI.create(URI.java:859)
at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:392)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:310)
at org.apache.oozie.util.JobUtils.addFileToClassPath(JobUtils.java:151)
at org.apache.oozie.service.HadoopAccessorService.run(HadoopAccessorService.java:539)
at org.apache.oozie.service.HadoopAccessorService.run(HadoopAccessorService.java:536)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
at org.apache.oozie.service.HadoopAccessorService.addFileToClassPath(HadoopAccessorService.java:536)
at org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:512)
... 12 more
Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 0: "maprfs:///"
at java.net.URI$Parser.fail(URI.java:2829)
at java.net.URI$Parser.checkChars(URI.java:3002)
at java.net.URI$Parser.checkChar(URI.java:3012)
at java.net.URI$Parser.parse(URI.java:3028)
at java.net.URI.<init>(URI.java:595)
at java.net.URI.create(URI.java:857)
... 23 more
2015-10-18 21:42:06,468 WARN ActionStartXCommand:544 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Setting Action Status to [DONE]
2015-10-18 21:42:06,494 INFO ActionEndXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] ERROR is considered as FAILED for SLA
2015-10-18 21:42:06,518 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] Start action [0000004-151018210741238-oozie-mapr-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,518 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] [***0000004-151018210741238-oozie-mapr-W@fail***]Action status=DONE
2015-10-18 21:42:06,518 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] [***0000004-151018210741238-oozie-mapr-W@fail***]Action updated in DB!
"
尽我所能找出问题的可能原因，但没有成功。与 mapr-oozie 包捆绑在一起的示例推荐使用 "maprfs:///" 作为 namenode 和 jobtracker。任何帮助将不胜感激。
我之前错过的问题是，当我在名称节点上收到错误时，由于 "job.properties" 文件的内容，错误是 而不是 。相反，它来自 "workflow.xml" 文件。我写错了：
<name-node>"${nameNode}"</name-node>
Oozie 不处理引号，如果删除引号，问题就会消失。最后的关键教训是，当您看到不容易解释的 Oozie 错误时，请小心检查 workflow.xml 和 job.properties 文件。
我在自定义 MapR 沙箱中 运行ning Oozie。下面是我的 job.properties 文件信息：
nameNode=maprfs:///
jobTracker=maprfs:///
queueName=default
examplesRoot=examples
oozie.wf.application.path=/user/user01/lab2/exercise4/zayd_hammoudeh_lab2_exercise4/workflow.xml
作为参考，下面是我的 workflow.xml 文件：
<action name="stdev-calc">
<map-reduce>
<job-tracker>${jobTracker}</job-tracker>
<name-node>"${nameNode}"</name-node>
<prepare>
<delete path="/user/user01/lab2/exercise4/OUT"/>
</prepare>
<configuration>
<property>
<name>mapred.mapper.class</name>
<value>Stdev.StdevMapper</value>
</property>
<property>
<name>mapred.reducer.class</name>
<value>Stdev.StdevReducer</value>
</property>
<property>
<name>mapred.input.dir</name>
<value>/user/user01/lab2/exercise4/DATA</value>
</property>
<property>
<name>mapred.output.dir</name>
<value>/user/user01/lab2/exercise4/OUT</value>
</property>
</configuration>
</map-reduce>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Exercise 4 failed with error message: [${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
当我尝试 运行 我的工作时，我在我的工作信息中得到以下信息：
Job ID : 0000004-151018210741238-oozie-mapr-W
------------------------------------------------------------------------------------------------------------------------------------
Workflow Name : aggregator-wf
App Path : /user/user01/lab2/exercise4/zayd_hammoudeh_lab2_exercise4/workflow.xml
Status : KILLED
Run : 0
User : user01
Group : -
Created : 2015-10-19 04:42 GMT
Started : 2015-10-19 04:42 GMT
Last Modified : 2015-10-19 04:42 GMT
Ended : 2015-10-19 04:42 GMT
CoordAction ID: -
Actions
------------------------------------------------------------------------------------------------------------------------------------
ID Status Ext ID Ext Status Err Code
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@:start: OK - OK -
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@stdev-calc ERROR - ERROR IllegalArgumentException
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@fail OK - OK E0729
------------------------------------------------------------------------------------------------------------------------------------
我的日志显示错误为：“[IllegalArgumentException：索引 0 处方案名称中的非法字符："maprfs:///"]”供参考，下面是我的日志：
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/mapr/oozie/oozie-4.1.0/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/mapr/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
2015-10-18 21:42:06,368 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] Start action [0000004-151018210741238-oozie-mapr-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,369 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] [***0000004-151018210741238-oozie-mapr-W@:start:***]Action status=DONE
2015-10-18 21:42:06,372 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] [***0000004-151018210741238-oozie-mapr-W@:start:***]Action updated in DB!
2015-10-18 21:42:06,413 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Start action [0000004-151018210741238-oozie-mapr-W@stdev-calc] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,468 WARN ActionStartXCommand:544 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Error starting action [stdev-calc]. ErrorType [ERROR], ErrorCode [IllegalArgumentException], Message [IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"]
org.apache.oozie.action.ActionExecutorException: IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"
at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:401)
at org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:532)
at org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:666)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:281)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"
at java.net.URI.create(URI.java:859)
at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:392)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:310)
at org.apache.oozie.util.JobUtils.addFileToClassPath(JobUtils.java:151)
at org.apache.oozie.service.HadoopAccessorService.run(HadoopAccessorService.java:539)
at org.apache.oozie.service.HadoopAccessorService.run(HadoopAccessorService.java:536)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
at org.apache.oozie.service.HadoopAccessorService.addFileToClassPath(HadoopAccessorService.java:536)
at org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:512)
... 12 more
Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 0: "maprfs:///"
at java.net.URI$Parser.fail(URI.java:2829)
at java.net.URI$Parser.checkChars(URI.java:3002)
at java.net.URI$Parser.checkChar(URI.java:3012)
at java.net.URI$Parser.parse(URI.java:3028)
at java.net.URI.<init>(URI.java:595)
at java.net.URI.create(URI.java:857)
... 23 more
2015-10-18 21:42:06,468 WARN ActionStartXCommand:544 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Setting Action Status to [DONE]
2015-10-18 21:42:06,494 INFO ActionEndXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] ERROR is considered as FAILED for SLA
2015-10-18 21:42:06,518 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] Start action [0000004-151018210741238-oozie-mapr-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,518 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] [***0000004-151018210741238-oozie-mapr-W@fail***]Action status=DONE
2015-10-18 21:42:06,518 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] [***0000004-151018210741238-oozie-mapr-W@fail***]Action updated in DB!
"
尽我所能找出问题的可能原因，但没有成功。与 mapr-oozie 包捆绑在一起的示例推荐使用 "maprfs:///" 作为 namenode 和 jobtracker。任何帮助将不胜感激。
我之前错过的问题是，当我在名称节点上收到错误时，由于 "job.properties" 文件的内容，错误是 而不是 。相反，它来自 "workflow.xml" 文件。我写错了：
<name-node>"${nameNode}"</name-node>
Oozie 不处理引号，如果删除引号，问题就会消失。最后的关键教训是，当您看到不容易解释的 Oozie 错误时，请小心检查 workflow.xml 和 job.properties 文件。