SAP HANA Vora：启动 SparkController 失败

Question

最近我开始在 AWS 上研究 SAP HANA Vora 1.0。我将 hanaes-site.xml 文件配置为读取 Vora 表，但是当尝试使用 ./hanaes start 命令启动控制器时，我收到此错误：

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/usr/sap/spark/controller/lib/spark-sap-datasources-1.0.0-assembly.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/usr/hdp/2.3.2.0-2950/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

16/04/21 10:29:31 INFO HanaESConfig: Loaded HANA Extended Store Configuration
Spark assembly is not available in class path. Creating Custom class loader from HDFS
Binding with hive-site.xml foind at /usr/sap/spark/controller/conf

16/04/21 10:29:34 INFO Server: Starting Spark Controller

16/04/21 10:29:34 INFO HanaESConfig: Loaded HANA Extended Store Configuration

16/04/21 10:29:51 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerBlockManagerAdded(1461234591659,BlockManagerId(2, master.cluster, 41198),1111511531)

你能帮我解决这个问题吗？

Answer 1

"Spark assembly is not available in class path"

您应该将 spark 程序集文件放在 /usr/sap/spark/controller/lib/external，并确保 hanaes.xml 具有属性:

<property>
 <name>sap.hana.es.spark.yarn.jar</name>
 <value>file:///usr/sap/spark/controller/lib/external/spark-assembly-<spark version>-hadoop<hadoop version>.jar</value>
 <final>true</final>
</property>

如果您的集群上已经安装了 Spark，您可以在 $SPARK_HOME/lib 找到 spark 程序集文件。如果您的集群上没有安装 Spark，您可以从 http://spark.apache.org/downloads.html

下载

谢谢

============更新=================

在 hanaes-site.xml 中，"sap.hana.es.lib.location" 应该指向 "hdfs:///sap/hana/spark/libs/thirdparty/"。

它应该如下所示：

<property>
 <name>sap.hana.es.spark.yarn.jar</name>
 <value>file:///usr/sap/spark/controller/lib/external/spark-assembly-1.4.1-hadoop2.6.0.jar</value>
 <final>true</final>
</property>
<property>
 <name>sap.hana.es.server.port</name>
 <value>7860</value>
 <final>true</final>
</property>
<property>
 <name>sap.hana.es.lib.location</name>
 <value>hdfs:///sap/hana/spark/libs/thirdparty/</value>
 <final>true</final>
</property>

此外，我也建议你使用Vora 1.2 AWS developer edition

SAP HANA Vora：启动 SparkController 失败

SAP HANA Vora: Failed to start SparkController

apache-spark

vora