Hortonworks shc 未解决的依赖项

Question

我想使用 hbase hortonworks 连接器。 github guide

但我不知道如何在我的项目中导入它。我有以下 build.sbt :

name := "project"

version := "1.0"

scalaVersion := "2.11.8"

libraryDependencies ++= Seq(
  "org.apache.spark" % "spark-core_2.11" % "2.2.0",
  "org.apache.spark" % "spark-sql_2.11" % "2.2.0",
  "org.scala-lang" % "scala-compiler" % "2.11.8",
  "com.hortonworks" % "shc" % "1.1.2-2.1-s_2.11-SNAPSHOT"
)

它给了我以下未解决的依赖关系：

Error:Error while importing SBT project:
...
[warn] ==== local: tried [warn] ivy.xml [warn] ==== public: tried [warn]
https://repo1.maven.org/maven2/com/hortonworks/shc/1.1.2-2.1-s_2.11-SNAPSHOT/shc-1.1.2-2.1-s_2.11-SNAPSHOT.pom [info] Resolving jline#jline;2.12.1 ... [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] ::
UNRESOLVED DEPENDENCIES :: [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: com.hortonworks#shc;1.1.2-2.1-s_2.11-SNAPSHOT: not found [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] [warn] Note: Unresolved dependencies path: [warn] com.hortonworks:shc:1.1.2-2.1-s_2.11-SNAPSHOT (build.sbt#L8-14) [warn] +- default:project_2.11:1.0 [trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output. [trace] Stack trace suppressed: run 'last :update' for the full output. [error] (:ssExtractDependencies) sbt.ResolveException: unresolved dependency: com.hortonworks#shc;1.1.2-2.1-s_2.11-SNAPSHOT: not found [error] (*:update) sbt.ResolveException: unresolved dependency: com.hortonworks#shc;1.1.2-2.1-s_2.11-SNAPSHOT: not found [error] Total time: 7 s, completed 3 août 2017 11:36:37

Answer 1

我的建议是不要使用最新版本的 Spark，因为所有最新版本都有一些错误需要修复。尝试使用 2.1.2 spark 版本而不是 2.2.0 并转到 Maven 存储库试试这个 link

在这里，您将获得 spark 的所有相关依赖项。

Answer 2

在您的解决方案中，您需要添加代表存储库的 resolvers 行。

scalaVersion := "2.11.12"

val sparkVersion = "2.1.0"

resolvers += "Hortonworks Repository" at "http://repo.hortonworks.com/content/repositories/releases/"

libraryDependencies += "org.apache.spark" %% "spark-core" % sparkVersion
libraryDependencies += "org.apache.spark" %% "spark-sql" % sparkVersion

libraryDependencies ++= Seq(
  "com.hortonworks" % "shc-core" % "1.1.1-2.1-s_2.11"
)

Hortonworks shc 未解决的依赖项

Hortonworks shc unresolved dependencies

hbase

sbt

hortonworks-data-platform