最短路径 - Giraph 示例 - 不适用于 AWS

Question

我遇到问题运行 AWS 上的最短路径示例。我通过 S3 下载了 giraph jar（在我正在使用的同一个 AMI 中编译并在那里上传），在主从中正确配置了 zookeper，我做了以下步骤：

纳米/tmp/tiny_graph.txt

[0,0,[1,1,3,3]] [1,0,[0,1,2,2,3,1]] [2,0,[1,2,4,4]] [3,0,[0,3,1,1,4,4]] [4,0,[3,4,2,4]]
hdfs dfs -mkdir /user/hduser
/home/hadoop/bin/hdfs dfs -copyFromLocal /tmp/tiny_graph.txt /user/tiny_graph.txt
hdfs dfs -mkdir /user/hduser/输出
/home/hadoop/bin/yarn jar /home/hadoop/share/hadoop/yarn/lib/giraph.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/hduser/input/ tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/输出/最短路径 -w 2

控制台向我显示了这个，但我不知道发生了什么：

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
SLF4J: Found binding in
SLF4J: Found binding in
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type http://org.slf4j.impl.Log4jLoggerFactory
16/04/17 04:44:46 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
16/04/17 04:44:46 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one.
16/04/17 04:44:46 INFO yarn.GiraphYarnClient: Final output path is: maprfs:/user/hduser/output/shortestpaths
16/04/17 04:44:46 INFO impl.TimelineClientImpl: Timeline service address: http://0.0.0.0:8188/ws/v1/timeline/
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.yarn.util.timeline.TimelineUtils.buildTimelineTokenService(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/io/Text;
at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:156)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.giraph.yarn.GiraphYarnClient.<init>(GiraphYarnClient.java:104)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:83)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

任何帮助将不胜感激！

Answer 1

我在 AWS 上停止使用 MapR 时解决了这个问题。由于某些原因 Giraph 和 MapR 不合得来。

最短路径 - Giraph 示例 - 不适用于 AWS

Shortest path - Giraph example - Not working on AWS

shortest-path

amazon-web-services

mapr

hadoop-yarn

giraph