如何 assemble Cassandra Connector for Spark for Scala 2.11？

Question

有人可以告诉我如何构建 Spark-Cassandra 连接器组件吗？我已尝试按照 Github 页面 https://github.com/datastax/spark-cassandra-connector 上的说明进行操作，但我只收到数百个 "deduplicate" 错误。

我将 Scala 2.11.7 与 Spark 1.5.1（我为 Scala 2.11 构建）和 SBT 13.8 一起使用。

我做了以下事情：

git clone https://github.com/datastax/spark-cassandra-connector.git
cd spark-cassandra-connector/
sbt -Dscala-2.11=true assembly

构建过程运行了一段时间，但随后开始吐出数百个 "deduplicate" 错误并失败。我不知道从哪里开始解决这个问题，但目前据我所知，这个项目的程序集构建过程不起作用。

关于如何解决这个问题的任何提示？

Answer 1

似乎 2.11 的构建已损坏，您应该将其报告给项目。不知道现在如何解决它。

➜  spark-cassandra-connector git:(master) sbt
[info] Loading global plugins from /Users/jacek/.sbt/0.13/plugins
[info] Loading project definition from /Users/jacek/dev/oss/spark-cassandra-connector/project
Using releases: https://oss.sonatype.org/service/local/staging/deploy/maven2 for releases
Using snapshots: https://oss.sonatype.org/content/repositories/snapshots for snapshots

  Scala: 2.10.5 [To build against Scala 2.11 use '-Dscala-2.11=true']
  Scala Binary: 2.10
  Java: target=1.7 user=1.8.0_66

[info] Set current project to root (in build file:/Users/jacek/dev/oss/spark-cassandra-connector/)
[root]> update
...
[info] Done updating.
[info] Done updating.
[success] Total time: 314 s, completed Dec 2, 2015 10:26:01 AM
[root]> 

➜  spark-cassandra-connector git:(master) sbt -Dscala-2.11=true
[info] Loading global plugins from /Users/jacek/.sbt/0.13/plugins
[info] Loading project definition from /Users/jacek/dev/oss/spark-cassandra-connector/project
Using releases: https://oss.sonatype.org/service/local/staging/deploy/maven2 for releases
Using snapshots: https://oss.sonatype.org/content/repositories/snapshots for snapshots

  Scala: 2.11.7
  Scala Binary: 2.11
  Java: target=1.7 user=1.8.0_66

[info] Set current project to root (in build file:/Users/jacek/dev/oss/spark-cassandra-connector/)
[root]> update
...
[error] impossible to get artifacts when data has not been loaded. IvyNode = org.slf4j#slf4j-log4j12;1.7.6
...
[trace] Stack trace suppressed: run last spark-cassandra-connector-embedded/*:update for the full output.
[error] (spark-cassandra-connector-embedded/*:update) java.lang.IllegalStateException: impossible to get artifacts when data has not been loaded. IvyNode = org.slf4j#slf4j-log4j12;1.7.6
[error] Total time: 9 s, completed Dec 2, 2015 10:27:19 AM

我提交了一个问题 https://datastax-oss.atlassian.net/browse/SPARKC-295。

如何 assemble Cassandra Connector for Spark for Scala 2.11？

How to assemble Cassandra Connector for Spark for Scala 2.11?

apache-spark

spark-cassandra-connector