Scala sbt assembly jar 不工作（class 未找到实现）但代码在通过 IntelliJ 时工作

Question

使用

启动我的代码时

scala -cp assembly.jar class.A --config-path confFile

我明白了

java.lang.IllegalStateException: No LivyClientFactory implementation was found

但是当通过 IntelliJ 启动时它工作正常。我还检查了我的装配罐，我得到了 LivyClientFactory 的 .class。

我怀疑是 build.sbt 错误，谁知道为什么他找不到 class?

我尝试尝试使用 assemblyMerge 策略但没有成功。

ThisBuild / scalaVersion := "2.12.10"
crossPaths := true
crossScalaVersions := Seq("2.12.10")
def resolveVersion(scalaV: String, versionsResolver: Map[String, String]): String = versionsResolver(scalaV.slice(0, 4))
val sparkVersions = Map("2.11" -> "2.4.3", "2.12" -> "3.0.1")
val scalaTestVersions = Map("2.11" -> "3.2.1", "2.12" -> "3.2.5")
val livyVersions = Map("2.11" -> "0.7.0-incubating", "2.12" -> "0.8.0-incubating")

// dependencies
val commonDependencies = settingKey[Seq[ModuleID]]("List of common dependencies across sub-projects.")
ThisBuild / commonDependencies := Seq(
  "org.apache.livy" % "livy-client-http" % resolveVersion(scalaVersion.value, livyVersions),
  "org.scalatest" %% "scalatest" % resolveVersion(scalaVersion.value,
                                                  scalaTestVersions
  ) % Test excludeAll excludeJbossNetty
)
libraryDependencies ++= commonDependencies.value ++ Seq(
  "com.typesafe.play" %% "play-json" % resolveVersion(scalaVersion.value, typeSafeVersions),
  "org.apache.httpcomponents" % "httpclient" % "4.5.12" % Test,
  "com.typesafe.scala-logging" %% "scala-logging" % "3.9.2",
  "info.picocli" % "picocli" % "4.1.1",
  "io.sentry" % "sentry-logback" % "1.7.16"
)
lazy val toBuild = (project in file("."))
  .enablePlugins(ScalaUnidocPlugin)
  .aggregate(submissions)
  .dependsOn(submissions)

lazy val submissions = project.settings(
  commonSettings,
  assemblySettings,
  libraryDependencies ++= commonDependencies.value ++ Seq(
    "org.apache.spark" %% "spark-core" % resolveVersion(scalaVersion.value, sparkVersions) % "provided",
    "org.apache.spark" %% "spark-sql" % resolveVersion(scalaVersion.value, sparkVersions) % "provided",
    "org.apache.spark" %% "spark-streaming" % resolveVersion(scalaVersion.value, sparkVersions) % "provided"
  )
)

lazy val commonResolvers = List()

lazy val assemblySettings = Seq(
  assemblyMergeStrategy in assembly := {
    case PathList("META-INF", xs @ _*) => MergeStrategy.discard
    case PathList(ps @ _*) if ps.last == "module-info.class" => MergeStrategy.discard
    case "module-info.class"           => MergeStrategy.discard
    case x =>
      val oldStrategy = (assemblyMergeStrategy in assembly).value
      oldStrategy(x)
  }
)
// Append Scala versions to the generated artifacts
crossPaths := true

// This forbids including Scala related libraries into the dependency
autoScalaLibrary := false

// Tests are executed sequentially
parallelExecution in Test := false
// path to scala doc with all modules
siteSubdirName in ScalaUnidoc := "api"
addMappingsToSiteDir(mappings in (ScalaUnidoc, packageDoc), siteSubdirName in ScalaUnidoc)

val excludeJbossNetty = ExclusionRule(organization = "org", "jboss.netty")

scalacOptions ++= Seq(
  "-encoding",
  "utf8",
  "-deprecation",
  "-feature",
  "-language:higherKinds",
  "-Ywarn-unused-import",
  "-Xfatal-warnings"
)
assemblyMergeStrategy in assembly := {
  case PathList("META-INF", xs @ _*) => MergeStrategy.discard
  case PathList(ps @ _*) if ps.last == "module-info.class" => MergeStrategy.discard
  case "module-info.class"           => MergeStrategy.discard
  case PathList("org", "aopalliance", xs @ _*)      => MergeStrategy.last
  case PathList("javax", "inject", xs @ _*)         => MergeStrategy.last
  case PathList("javax", "servlet", xs @ _*)        => MergeStrategy.last
  case PathList("javax", "activation", xs @ _*)     => MergeStrategy.last
  case PathList("org", "apache", xs @ _*)           => MergeStrategy.last
  case PathList("com", "google", xs @ _*)           => MergeStrategy.last
  case PathList("com", "esotericsoftware", xs @ _*) => MergeStrategy.last
  case PathList("com", "codahale", xs @ _*)         => MergeStrategy.last
  case PathList("com", "yammer", xs @ _*)           => MergeStrategy.last
  case "about.html"                                 => MergeStrategy.rename
  case "META-INF/DISCLAIMER"                        => MergeStrategy.last
  case "META-INF/ECLIPSEF.RSA"                      => MergeStrategy.last
  case "META-INF/mailcap"                           => MergeStrategy.last
  case "META-INF/mimetypes.default"                 => MergeStrategy.last
  case "plugin.properties"                          => MergeStrategy.last
  case "log4j.properties"                           => MergeStrategy.last
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}
artifact in (Compile, assembly) := {
  val art = (artifact in (Compile, assembly)).value
  art.withClassifier(Some("assembly"))
}
addArtifact(artifact in (Compile, assembly), assembly)

Answer 1

该 jar 在 META-INF 中缺少服务

  case PathList("META-INF", "services", xs @ _*) => MergeStrategy.first
  case PathList("META-INF", xs @ _*) => MergeStrategy.discard
  case PathList(ps @ _*) if ps.last == "module-info.class" => MergeStrategy.discard
  case "module-info.class"           => MergeStrategy.discard

Scala sbt assembly jar 不工作（class 未找到实现）但代码在通过 IntelliJ 时工作

Scala sbt assembly jar does not work (class implementation not found) but code works when through IntelliJ

scala

intellij-idea

sbt

sbt-assembly

livy