SBT 组装失败
SBT assembly falis
我正在 运行通过 intellij 进行 spark 作业。作业执行并给我输出。我需要将此作业作为 jar 文件发送到服务器并 运行,但是当我尝试执行 sbt assembly
时,它会抛出以下错误:
[error] Not a valid command: assembly
[error] Not a valid project ID: assembly
[error] Expected ':' (if selecting a configuration)
[error] Not a valid key: assembly
[error] assembly
我的 sbt 版本是 0.13.8
下面是我的 build.sbt 文件:
import sbt._, Keys._
name := "mobilewalla"
version := "1.0"
scalaVersion := "2.11.7"
libraryDependencies ++= Seq("org.apache.spark" %% "spark-core" % "2.0.0",
"org.apache.spark" %% "spark-sql" % "2.0.0")
我在项目目录下添加了一个文件assembly.sbt。它包含:
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.3")
我在这里错过了什么
为了assemble 将多个jar 变成一个你需要在项目目录下的plugins.sbt 中添加下面的插件。
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.3")
如果您需要自定义 assembled jar 以触发特定的 MainClass 以示例 assembly.sbt
import sbtassembly.Plugin.AssemblyKeys._
Project.inConfig(Compile)(baseAssemblySettings)
mainClass in (Compile, assembly) := Some("<main application name with package path>")
jarName in (Compile, assembly) := s"${name.value}-${version.value}-dist.jar"
//below is merge strategy to make what all file need to exclude or include
mergeStrategy in (Compile, assembly) <<= (mergeStrategy in (Compile, assembly)) {
(old) => {
case PathList(ps @ _*) if ps.last endsWith ".html" =>MergeStrategy.first
case "META-INF/MANIFEST.MF" => MergeStrategy.discard
case x => old(x)
}
}
将这些行添加到您的 build.sbt
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs @ _*) => MergeStrategy.discard
case x => MergeStrategy.first
}
mainClass in assembly := Some("com.SparkMain")
resolvers += "spray repo" at "http://repo.spray.io"
assemblyJarName in assembly := "streaming-api.jar"
并将这些行包含在您的 plugins.sbt 文件中
addSbtPlugin("io.spray" % "sbt-revolver" % "0.7.2")
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.13.0")
我正在 运行通过 intellij 进行 spark 作业。作业执行并给我输出。我需要将此作业作为 jar 文件发送到服务器并 运行,但是当我尝试执行 sbt assembly
时,它会抛出以下错误:
[error] Not a valid command: assembly
[error] Not a valid project ID: assembly
[error] Expected ':' (if selecting a configuration)
[error] Not a valid key: assembly
[error] assembly
我的 sbt 版本是 0.13.8
下面是我的 build.sbt 文件:
import sbt._, Keys._
name := "mobilewalla"
version := "1.0"
scalaVersion := "2.11.7"
libraryDependencies ++= Seq("org.apache.spark" %% "spark-core" % "2.0.0",
"org.apache.spark" %% "spark-sql" % "2.0.0")
我在项目目录下添加了一个文件assembly.sbt。它包含:
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.3")
我在这里错过了什么
为了assemble 将多个jar 变成一个你需要在项目目录下的plugins.sbt 中添加下面的插件。
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.3")
如果您需要自定义 assembled jar 以触发特定的 MainClass 以示例 assembly.sbt
import sbtassembly.Plugin.AssemblyKeys._
Project.inConfig(Compile)(baseAssemblySettings)
mainClass in (Compile, assembly) := Some("<main application name with package path>")
jarName in (Compile, assembly) := s"${name.value}-${version.value}-dist.jar"
//below is merge strategy to make what all file need to exclude or include
mergeStrategy in (Compile, assembly) <<= (mergeStrategy in (Compile, assembly)) {
(old) => {
case PathList(ps @ _*) if ps.last endsWith ".html" =>MergeStrategy.first
case "META-INF/MANIFEST.MF" => MergeStrategy.discard
case x => old(x)
}
}
将这些行添加到您的 build.sbt
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs @ _*) => MergeStrategy.discard
case x => MergeStrategy.first
}
mainClass in assembly := Some("com.SparkMain")
resolvers += "spray repo" at "http://repo.spray.io"
assemblyJarName in assembly := "streaming-api.jar"
并将这些行包含在您的 plugins.sbt 文件中
addSbtPlugin("io.spray" % "sbt-revolver" % "0.7.2")
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.13.0")