flink:scala 版本冲突?

flink: scala version conflict?

我正在尝试在 IntelliJ 中编译来自 here 的 kafka 样本。在对依赖项大惊小怪之后 运行 进入了这个我似乎无法解决的问题:

15/10/25 12:36:34 ERROR actor.ActorSystemImpl: Uncaught fatal error from thread [flink-akka.actor.default-dispatcher-4] shutting down ActorSystem [flink]
java.lang.NoClassDefFoundError: scala/runtime/AbstractPartialFunction$mcVL$sp
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:455)
at java.net.URLClassLoader.access0(URLClassLoader.java:73)
at java.net.URLClassLoader.run(URLClassLoader.java:367)
at java.net.URLClassLoader.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.flink.runtime.jobmanager.MemoryArchivist.handleMessage(MemoryArchivist.scala:80)
at org.apache.flink.runtime.FlinkActor$class.receive(FlinkActor.scala:32)
at org.apache.flink.runtime.jobmanager.MemoryArchivist.org$apache$flink$runtime$LogMessages$$super$receive(MemoryArchivist.scala:59)
at org.apache.flink.runtime.LogMessages$class.receive(LogMessages.scala:26)
at org.apache.flink.runtime.jobmanager.MemoryArchivist.receive(MemoryArchivist.scala:59)
at akka.actor.ActorCell.newActor(ActorCell.scala:567)
at akka.actor.ActorCell.create(ActorCell.scala:587)
at akka.actor.ActorCell.invokeAll(ActorCell.scala:460)
at akka.actor.ActorCell.systemInvoke(ActorCell.scala:482)
at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:282)
at akka.dispatch.Mailbox.run(Mailbox.scala:223)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)


Caused by: java.lang.ClassNotFoundException: scala.runtime.AbstractPartialFunction$mcVL$sp
at java.net.URLClassLoader.run(URLClassLoader.java:372)
at java.net.URLClassLoader.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more

我 运行 有一些观点表明这是 Scala 版本的问题。当前图书馆列表:

flink-runtime-1.0-SNAPSHOT
flink-streaming-java-1.0-SNAPSHOT
flink-connector-kafka-1.0-SNAPSHOT
flink-java8-1.0-SNAPSHOT
flink-core-1.0-SNAPSHOT
flink-java-1.0-SNAPSHOT
org.apache.hadoop:hadoop-core:1.2.1
flink-clients-1.0-SNAPSHOT
org.apache.kafka:kafka-clients:0.8.2.2
org.apache.kafka:kafka_2.11:0.8.2.2
flink-optimizer-1.0-SNAPSHOT
org.apache.sling:org.apache.sling.commons.json:2.0.6
de.javakaffee:kryo-serializers:0.28
com.github.scopt:scopt_2.11:3.3.0
org.clapper:grizzled-slf4j_2.9.0:0.6.6
com.typesafe.akka:akka-osgi_2.11:2.4.0
com.typesafe.akka:akka-slf4j_2.11:2.4.0

关于我运行误入歧途的地方的建议?

确实是Scala版本不匹配的问题。您正在混合为 Scala 2.11 构建的依赖项,例如org.apache.kafka:kafka_2.11:0.8.2.2 具有默认为 Scala 构建的 Flink 依赖项 2.10

为 Scala 2.11 构建的依赖项之一引入了 scala-library:2.11 jar,它替换了 Flink 依赖项所需的 scala-library:2.10 依赖项。您可以使用为 Scala 2.10 构建的二进制文件作为非 Flink 依赖项,或者使用 Scala 2.11 构建和安装 Flink。有关如何使用不同的 Scala 版本构建 Flink,请参阅 https://ci.apache.org/projects/flink/flink-docs-master/setup/building.html#build-flink-for-a-specific-scala-version

Kafka 示例

如果你只想将引用的 Kafka 示例的版本升级到 0.10-SNAPSHOT,你必须更改 pom.xml 文件中的 Flink 版本,你必须使用 FlinkKafkaProducer而不是 WriteIntoKafka.java 文件中的 KafkaSink。你不再需要 SimpleStringSchema 了。这就是您需要更改的全部内容(不需要额外的依赖项)。