运行 烫烫作业时出现 NoSuchMethodError
NoSuchMethodError when running scalding job
我正在尝试将 CDH 4.5 上的烫金作业 运行ning 升级到 CDH 5.5.1。
该作业使用 json4s 来解析 json 数据。当我 运行 集群上的作业时，出现以下错误。它在我的本地机器上运行良好。
我正在使用 scala 2.10 和 Scalding 0.15.0 版本。
ERROR [main] cascading.flow.stream.SourceStage: caught throwable
cascading.pipe.OperatorException: [JsonLine(/data/raw/gon...][read() @ com.xxx.xxx.GJob.<init>(GonzalesJob.scala:15)] operator Each failed executing operation
at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:107)
at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:39)
at cascading.flow.stream.SourceStage.map(SourceStage.java:102)
at cascading.flow.stream.SourceStage.run(SourceStage.java:58)
at cascading.flow.hadoop.FlowMapper.run(FlowMapper.java:130)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.deser.std.ContainerDeserializerBase: method <init>(Lcom/fasterxml/jackson/databind/JavaType;)V not found
at com.fasterxml.jackson.module.scala.deser.UnsortedMapDeserializer.<init>(UnsortedMapDeserializerModule.scala:39)
at com.fasterxml.jackson.module.scala.deser.UnsortedMapDeserializerResolver$.findMapLikeDeserializer(UnsortedMapDeserializerModule.scala:97)
at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._findCustomMapLikeDeserializer(BasicDeserializerFactory.java:1030)
at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.createMapLikeDeserializer(BasicDeserializerFactory.java:994)
at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:384)
at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:354)
at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:267)
at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:247)
at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:146)
at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:322)
at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:2990)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2884)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2041)
at com.twitter.scalding.JsonLine$$anonfun$transformForRead.applyOrElse(JsonLine.scala:70)
at com.twitter.scalding.JsonLine$$anonfun$transformForRead.applyOrElse(JsonLine.scala:68)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33)
at com.twitter.scalding.CollectFunction.operate(Operations.scala:94)
at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:99)
... 11 more
这是我依赖项中 jackson 的 grep
mvn dependency:tree
--- maven-dependency-plugin:2.8:tree (default-cli) @ ---
[INFO] bskyb.bigdata:gonzales-etl:jar:1.3
[INFO] +- org.scala-lang:scala-library:jar:2.10.5:compile
[INFO] +- org.scalatest:scalatest_2.10:jar:2.2.2:test
[INFO] | \- org.scala-lang:scala-reflect:jar:2.10.4:compile
[INFO] +- org.apache.hadoop:hadoop-yarn-client:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- com.google.guava:guava:jar:11.0.2:compile
[INFO] | | \- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] | +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] | +- commons-lang:commons-lang:jar:2.6:compile
[INFO] | +- commons-cli:commons-cli:jar:1.2:compile
[INFO] | +- log4j:log4j:jar:1.2.17:compile
[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- org.apache.hadoop:hadoop-yarn-api:jar:2.6.0-cdh5.5.1:compile
[INFO] | | \- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] | \- org.apache.hadoop:hadoop-yarn-common:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- javax.xml.bind:jaxb-api:jar:2.2.2:compile
[INFO] | | +- javax.xml.stream:stax-api:jar:1.0-2:compile
[INFO] | | \- javax.activation:activation:jar:1.1:compile
[INFO] | +- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] | | \- org.tukaani:xz:jar:1.0:compile
[INFO] | +- commons-codec:commons-codec:jar:1.4:compile
[INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26.cloudera.4:compile
[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile
[INFO] | +- com.sun.jersey:jersey-client:jar:1.9:compile
[INFO] | +- com.google.inject.extensions:guice-servlet:jar:3.0:compile
[INFO] | +- commons-io:commons-io:jar:2.4:compile
[INFO] | +- com.google.inject:guice:jar:3.0:compile
[INFO] | | +- javax.inject:javax.inject:jar:1:compile
[INFO] | | \- aopalliance:aopalliance:jar:1.0:compile
[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile
[INFO] | | \- asm:asm:jar:3.1:compile
[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile
[INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:compile
[INFO] | | \- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO] | \- com.sun.jersey.contribs:jersey-guice:jar:1.9:compile
[INFO] +- com.twitter:scalding-commons_2.10:jar:0.11.0:compile
[INFO] | +- com.twitter:scalding-args_2.10:jar:0.11.0:compile
[INFO] | +- com.backtype:dfs-datastores-cascading:jar:1.3.4:compile
[INFO] | +- com.backtype:dfs-datastores:jar:1.3.4:compile
[INFO] | | \- jvyaml:jvyaml:jar:1.0.0:compile
[INFO] | +- com.twitter:bijection-core_2.10:jar:0.6.3:compile
[INFO] | +- com.twitter:algebird-core_2.10:jar:0.7.0:compile
[INFO] | | \- com.googlecode.javaewah:JavaEWAH:jar:0.6.6:compile
[INFO] | +- com.twitter:chill_2.10:jar:0.4.0:compile
[INFO] | | \- com.esotericsoftware.kryo:kryo:jar:2.21:compile
[INFO] | | +- com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07:compile
[INFO] | | | \- org.ow2.asm:asm:jar:4.0:compile
[INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile
[INFO] | | \- org.objenesis:objenesis:jar:1.2:compile
[INFO] | +- com.twitter.elephantbird:elephant-bird-cascading2:jar:4.4:compile
[INFO] | +- com.hadoop.gplcompression:hadoop-lzo:jar:0.4.16:compile
[INFO] | +- org.apache.thrift:libthrift:jar:0.5.0:compile
[INFO] | \- org.slf4j:slf4j-api:jar:1.6.6:compile
[INFO] +- com.twitter:scalding-core_2.10:jar:0.15.0:compile
[INFO] | +- com.twitter:scalding-serialization_2.10:jar:0.15.0:compile
[INFO] | +- com.twitter:maple:jar:0.15.0:compile
[INFO] | +- cascading:cascading-core:jar:2.6.1:compile
[INFO] | | +- riffle:riffle:jar:0.1-dev:compile
[INFO] | | +- thirdparty:jgrapht-jdk1.6:jar:0.8.1:compile
[INFO] | | \- org.codehaus.janino:janino:jar:2.7.5:compile
[INFO] | | \- org.codehaus.janino:commons-compiler:jar:2.7.5:compile
[INFO] | +- cascading:cascading-local:jar:2.6.1:compile
[INFO] | +- cascading:cascading-hadoop:jar:2.6.1:compile
[INFO] | +- com.twitter:chill-hadoop:jar:0.6.0:compile
[INFO] | +- com.twitter:chill-algebird_2.10:jar:0.6.0:compile
[INFO] | \- com.twitter:chill-java:jar:0.6.0:compile
[INFO] +- com.twitter:scalding-date_2.10:jar:0.15.0:compile
[INFO] +- com.twitter:scalding-json_2.10:jar:0.15.0:compile
**[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.10:jar:2.4.2:compile**
[INFO] | | \- com.thoughtworks.paranamer:paranamer:jar:2.6:compile
[INFO] | \- org.json4s:json4s-native_2.10:jar:3.2.11:compile
[INFO] +- parallelai:parallelai.spyglass:jar:2.10_0.10_CDH5.2.1_4.4:compile
[INFO] | +- org.apache.hadoop:hadoop-client:jar:2.5.0-cdh5.2.1:compile
[INFO] | | +- org.apache.hadoop:hadoop-common:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | +- org.apache.commons:commons-math3:jar:3.1.1:compile
[INFO] | | | +- xmlenc:xmlenc:jar:0.52:compile
[INFO] | | | +- commons-net:commons-net:jar:3.1:compile
[INFO] | | | +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | | | | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] | | | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] | | | +- org.slf4j:slf4j-log4j12:jar:1.7.5:compile
[INFO] | | | +- org.apache.avro:avro:jar:1.7.6-cdh5.2.1:compile
[INFO] | | | | \- org.xerial.snappy:snappy-java:jar:1.0.5:compile
[INFO] | | | +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] | | | +- org.apache.curator:curator-client:jar:2.6.0:compile
[INFO] | | | \- org.apache.curator:curator-recipes:jar:2.6.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-hdfs:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | \- io.netty:netty:jar:3.6.2.Final:compile
[INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | | \- org.apache.hadoop:hadoop-yarn-server-common:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile
[INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.5.0-cdh5.2.1:compile
[INFO] | | \- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.5.0-cdh5.2.1:compile
[INFO] | +- org.apache.hbase:hbase-server:jar:0.98.6-cdh5.2.1:compile
[INFO] | | +- org.apache.hbase:hbase-common:jar:0.98.6-cdh5.2.1:compile
[INFO] | | +- org.apache.hbase:hbase-protocol:jar:0.98.6-cdh5.2.1:compile
[INFO] | | +- org.apache.hbase:hbase-client:jar:0.98.6-cdh5.2.1:compile
[INFO] | | +- org.apache.hbase:hbase-prefix-tree:jar:0.98.6-cdh5.2.1:runtime
[INFO] | | | \- org.apache.hbase:hbase-common:jar:tests:0.98.6-cdh5.2.1:runtime
[INFO] | | +- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | | +- commons-collections:commons-collections:jar:3.2.1:compile
[INFO] | | +- org.apache.hbase:hbase-hadoop-compat:jar:0.98.6-cdh5.2.1:compile
[INFO] | | +- org.apache.hbase:hbase-hadoop2-compat:jar:0.98.6-cdh5.2.1:compile
[INFO] | | +- com.yammer.metrics:metrics-core:jar:2.2.0:compile
[INFO] | | +- com.github.stephenc.high-scale-lib:high-scale-lib:jar:1.1.1:compile
[INFO] | | +- org.apache.commons:commons-math:jar:2.1:compile
[INFO] | | +- org.apache.zookeeper:zookeeper:jar:3.4.5-cdh5.2.1:compile
[INFO] | | +- org.mortbay.jetty:jetty:jar:6.1.26.cloudera.4:compile
[INFO] | | +- org.mortbay.jetty:jetty-sslengine:jar:6.1.26.cloudera.4:compile
[INFO] | | +- org.mortbay.jetty:jsp-2.1:jar:6.1.14:compile
[INFO] | | +- org.mortbay.jetty:jsp-api-2.1:jar:6.1.14:compile
[INFO] | | +- tomcat:jasper-compiler:jar:5.5.23:compile
[INFO] | | +- tomcat:jasper-runtime:jar:5.5.23:compile
[INFO] | | | \- commons-el:commons-el:jar:1.0:compile
[INFO] | | +- org.jamon:jamon-runtime:jar:2.3.1:compile
[INFO] | | +- org.cloudera.htrace:htrace-core:jar:2.04:compile
[INFO] | | +- org.apache.hadoop:hadoop-auth:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile
[INFO] | | | | \- org.apache.httpcomponents:httpcore:jar:4.2.4:compile
[INFO] | | | +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile
[INFO] | | | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile
[INFO] | | | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile
[INFO] | | | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile
[INFO] | | | \- org.apache.curator:curator-framework:jar:2.6.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-core:jar:2.5.0-mr1-cdh5.2.1:compile
[INFO] | | | +- javax.servlet.jsp:jsp-api:jar:2.1:compile
[INFO] | | | +- net.java.dev.jets3t:jets3t:jar:0.6.1:compile
[INFO] | | | +- hsqldb:hsqldb:jar:1.8.0.10:compile
[INFO] | | | \- org.eclipse.jdt:core:jar:3.1.1:compile
[INFO] | | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:2.5.0-cdh5.2.1:compile
[INFO] | | | \- commons-daemon:commons-daemon:jar:1.0.13:compile
[INFO] | | +- com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1:compile
[INFO] | | \- junit:junit:jar:4.11:compile
[INFO] | | \- org.hamcrest:hamcrest-core:jar:1.3:compile
[INFO] | +- com.jcraft:jsch:jar:0.1.51:compile
[INFO] | +- com.twitter.elephantbird:elephant-bird-core:jar:4.1:compile
[INFO] | | \- com.googlecode.json-simple:json-simple:jar:1.1:compile
[INFO] | \- com.twitter.elephantbird:elephant-bird-hadoop-compat:jar:4.1:compile
[INFO] +- com.typesafe:config:jar:1.2.1:compile
[INFO] +- org.json4s:json4s-jackson_2.10:jar:3.3.0.RC1:compile
[INFO] | \- org.json4s:json4s-core_2.10:jar:3.3.0.RC1:compile
[INFO] | +- org.json4s:json4s-ast_2.10:jar:3.3.0.RC1:compile
[INFO] | \- org.json4s:json4s-scalap_2.10:jar:3.3.0.RC1:compile
**[INFO] \- com.fasterxml.jackson.core:jackson-databind:jar:2.4.2:compile
[INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.4.0:compile
[INFO] \- com.fasterxml.jackson.core:jackson-core:jar:2.4.2:compile**
NoSuchMethodError 始终表示不兼容的 jar 文件。在你的情况下，我会说它在 jackson.module.scala 和 jackson.databind
之间
请切换到两个文件的相同版本。
Hadoop 通常在类路径上有许多 jar 的自己的版本。您可以尝试通过将 mapreduce.job.user.classpath.first 设置为 true 来设置您的类路径以覆盖它。或者，您可以降级您的项目以使用与您的 hadoop 部署所使用的相同版本的 jackson。
非隔离类路径可能是 hadoop 最大的痛点。
我正在尝试将 CDH 4.5 上的烫金作业 运行ning 升级到 CDH 5.5.1。 该作业使用 json4s 来解析 json 数据。当我 运行 集群上的作业时，出现以下错误。它在我的本地机器上运行良好。
我正在使用 scala 2.10 和 Scalding 0.15.0 版本。
ERROR [main] cascading.flow.stream.SourceStage: caught throwable
cascading.pipe.OperatorException: [JsonLine(/data/raw/gon...][read() @ com.xxx.xxx.GJob.<init>(GonzalesJob.scala:15)] operator Each failed executing operation
at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:107)
at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:39)
at cascading.flow.stream.SourceStage.map(SourceStage.java:102)
at cascading.flow.stream.SourceStage.run(SourceStage.java:58)
at cascading.flow.hadoop.FlowMapper.run(FlowMapper.java:130)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.deser.std.ContainerDeserializerBase: method <init>(Lcom/fasterxml/jackson/databind/JavaType;)V not found
at com.fasterxml.jackson.module.scala.deser.UnsortedMapDeserializer.<init>(UnsortedMapDeserializerModule.scala:39)
at com.fasterxml.jackson.module.scala.deser.UnsortedMapDeserializerResolver$.findMapLikeDeserializer(UnsortedMapDeserializerModule.scala:97)
at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._findCustomMapLikeDeserializer(BasicDeserializerFactory.java:1030)
at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.createMapLikeDeserializer(BasicDeserializerFactory.java:994)
at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:384)
at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:354)
at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:267)
at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:247)
at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:146)
at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:322)
at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:2990)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2884)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2041)
at com.twitter.scalding.JsonLine$$anonfun$transformForRead.applyOrElse(JsonLine.scala:70)
at com.twitter.scalding.JsonLine$$anonfun$transformForRead.applyOrElse(JsonLine.scala:68)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33)
at com.twitter.scalding.CollectFunction.operate(Operations.scala:94)
at cascading.flow.stream.FunctionEachStage.receive(FunctionEachStage.java:99)
... 11 more
这是我依赖项中 jackson 的 grep
mvn dependency:tree
--- maven-dependency-plugin:2.8:tree (default-cli) @ ---
[INFO] bskyb.bigdata:gonzales-etl:jar:1.3
[INFO] +- org.scala-lang:scala-library:jar:2.10.5:compile
[INFO] +- org.scalatest:scalatest_2.10:jar:2.2.2:test
[INFO] | \- org.scala-lang:scala-reflect:jar:2.10.4:compile
[INFO] +- org.apache.hadoop:hadoop-yarn-client:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- com.google.guava:guava:jar:11.0.2:compile
[INFO] | | \- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] | +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] | +- commons-lang:commons-lang:jar:2.6:compile
[INFO] | +- commons-cli:commons-cli:jar:1.2:compile
[INFO] | +- log4j:log4j:jar:1.2.17:compile
[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- org.apache.hadoop:hadoop-yarn-api:jar:2.6.0-cdh5.5.1:compile
[INFO] | | \- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] | \- org.apache.hadoop:hadoop-yarn-common:jar:2.6.0-cdh5.5.1:compile
[INFO] | +- javax.xml.bind:jaxb-api:jar:2.2.2:compile
[INFO] | | +- javax.xml.stream:stax-api:jar:1.0-2:compile
[INFO] | | \- javax.activation:activation:jar:1.1:compile
[INFO] | +- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] | | \- org.tukaani:xz:jar:1.0:compile
[INFO] | +- commons-codec:commons-codec:jar:1.4:compile
[INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26.cloudera.4:compile
[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile
[INFO] | +- com.sun.jersey:jersey-client:jar:1.9:compile
[INFO] | +- com.google.inject.extensions:guice-servlet:jar:3.0:compile
[INFO] | +- commons-io:commons-io:jar:2.4:compile
[INFO] | +- com.google.inject:guice:jar:3.0:compile
[INFO] | | +- javax.inject:javax.inject:jar:1:compile
[INFO] | | \- aopalliance:aopalliance:jar:1.0:compile
[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile
[INFO] | | \- asm:asm:jar:3.1:compile
[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile
[INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:compile
[INFO] | | \- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO] | \- com.sun.jersey.contribs:jersey-guice:jar:1.9:compile
[INFO] +- com.twitter:scalding-commons_2.10:jar:0.11.0:compile
[INFO] | +- com.twitter:scalding-args_2.10:jar:0.11.0:compile
[INFO] | +- com.backtype:dfs-datastores-cascading:jar:1.3.4:compile
[INFO] | +- com.backtype:dfs-datastores:jar:1.3.4:compile
[INFO] | | \- jvyaml:jvyaml:jar:1.0.0:compile
[INFO] | +- com.twitter:bijection-core_2.10:jar:0.6.3:compile
[INFO] | +- com.twitter:algebird-core_2.10:jar:0.7.0:compile
[INFO] | | \- com.googlecode.javaewah:JavaEWAH:jar:0.6.6:compile
[INFO] | +- com.twitter:chill_2.10:jar:0.4.0:compile
[INFO] | | \- com.esotericsoftware.kryo:kryo:jar:2.21:compile
[INFO] | | +- com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07:compile
[INFO] | | | \- org.ow2.asm:asm:jar:4.0:compile
[INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile
[INFO] | | \- org.objenesis:objenesis:jar:1.2:compile
[INFO] | +- com.twitter.elephantbird:elephant-bird-cascading2:jar:4.4:compile
[INFO] | +- com.hadoop.gplcompression:hadoop-lzo:jar:0.4.16:compile
[INFO] | +- org.apache.thrift:libthrift:jar:0.5.0:compile
[INFO] | \- org.slf4j:slf4j-api:jar:1.6.6:compile
[INFO] +- com.twitter:scalding-core_2.10:jar:0.15.0:compile
[INFO] | +- com.twitter:scalding-serialization_2.10:jar:0.15.0:compile
[INFO] | +- com.twitter:maple:jar:0.15.0:compile
[INFO] | +- cascading:cascading-core:jar:2.6.1:compile
[INFO] | | +- riffle:riffle:jar:0.1-dev:compile
[INFO] | | +- thirdparty:jgrapht-jdk1.6:jar:0.8.1:compile
[INFO] | | \- org.codehaus.janino:janino:jar:2.7.5:compile
[INFO] | | \- org.codehaus.janino:commons-compiler:jar:2.7.5:compile
[INFO] | +- cascading:cascading-local:jar:2.6.1:compile
[INFO] | +- cascading:cascading-hadoop:jar:2.6.1:compile
[INFO] | +- com.twitter:chill-hadoop:jar:0.6.0:compile
[INFO] | +- com.twitter:chill-algebird_2.10:jar:0.6.0:compile
[INFO] | \- com.twitter:chill-java:jar:0.6.0:compile
[INFO] +- com.twitter:scalding-date_2.10:jar:0.15.0:compile
[INFO] +- com.twitter:scalding-json_2.10:jar:0.15.0:compile
**[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.10:jar:2.4.2:compile**
[INFO] | | \- com.thoughtworks.paranamer:paranamer:jar:2.6:compile
[INFO] | \- org.json4s:json4s-native_2.10:jar:3.2.11:compile
[INFO] +- parallelai:parallelai.spyglass:jar:2.10_0.10_CDH5.2.1_4.4:compile
[INFO] | +- org.apache.hadoop:hadoop-client:jar:2.5.0-cdh5.2.1:compile
[INFO] | | +- org.apache.hadoop:hadoop-common:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | +- org.apache.commons:commons-math3:jar:3.1.1:compile
[INFO] | | | +- xmlenc:xmlenc:jar:0.52:compile
[INFO] | | | +- commons-net:commons-net:jar:3.1:compile
[INFO] | | | +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | | | | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] | | | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] | | | +- org.slf4j:slf4j-log4j12:jar:1.7.5:compile
[INFO] | | | +- org.apache.avro:avro:jar:1.7.6-cdh5.2.1:compile
[INFO] | | | | \- org.xerial.snappy:snappy-java:jar:1.0.5:compile
[INFO] | | | +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] | | | +- org.apache.curator:curator-client:jar:2.6.0:compile
[INFO] | | | \- org.apache.curator:curator-recipes:jar:2.6.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-hdfs:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | \- io.netty:netty:jar:3.6.2.Final:compile
[INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | | \- org.apache.hadoop:hadoop-yarn-server-common:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile
[INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.5.0-cdh5.2.1:compile
[INFO] | | \- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.5.0-cdh5.2.1:compile
[INFO] | +- org.apache.hbase:hbase-server:jar:0.98.6-cdh5.2.1:compile
[INFO] | | +- org.apache.hbase:hbase-common:jar:0.98.6-cdh5.2.1:compile
[INFO] | | +- org.apache.hbase:hbase-protocol:jar:0.98.6-cdh5.2.1:compile
[INFO] | | +- org.apache.hbase:hbase-client:jar:0.98.6-cdh5.2.1:compile
[INFO] | | +- org.apache.hbase:hbase-prefix-tree:jar:0.98.6-cdh5.2.1:runtime
[INFO] | | | \- org.apache.hbase:hbase-common:jar:tests:0.98.6-cdh5.2.1:runtime
[INFO] | | +- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | | +- commons-collections:commons-collections:jar:3.2.1:compile
[INFO] | | +- org.apache.hbase:hbase-hadoop-compat:jar:0.98.6-cdh5.2.1:compile
[INFO] | | +- org.apache.hbase:hbase-hadoop2-compat:jar:0.98.6-cdh5.2.1:compile
[INFO] | | +- com.yammer.metrics:metrics-core:jar:2.2.0:compile
[INFO] | | +- com.github.stephenc.high-scale-lib:high-scale-lib:jar:1.1.1:compile
[INFO] | | +- org.apache.commons:commons-math:jar:2.1:compile
[INFO] | | +- org.apache.zookeeper:zookeeper:jar:3.4.5-cdh5.2.1:compile
[INFO] | | +- org.mortbay.jetty:jetty:jar:6.1.26.cloudera.4:compile
[INFO] | | +- org.mortbay.jetty:jetty-sslengine:jar:6.1.26.cloudera.4:compile
[INFO] | | +- org.mortbay.jetty:jsp-2.1:jar:6.1.14:compile
[INFO] | | +- org.mortbay.jetty:jsp-api-2.1:jar:6.1.14:compile
[INFO] | | +- tomcat:jasper-compiler:jar:5.5.23:compile
[INFO] | | +- tomcat:jasper-runtime:jar:5.5.23:compile
[INFO] | | | \- commons-el:commons-el:jar:1.0:compile
[INFO] | | +- org.jamon:jamon-runtime:jar:2.3.1:compile
[INFO] | | +- org.cloudera.htrace:htrace-core:jar:2.04:compile
[INFO] | | +- org.apache.hadoop:hadoop-auth:jar:2.5.0-cdh5.2.1:compile
[INFO] | | | +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile
[INFO] | | | | \- org.apache.httpcomponents:httpcore:jar:4.2.4:compile
[INFO] | | | +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile
[INFO] | | | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile
[INFO] | | | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile
[INFO] | | | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile
[INFO] | | | \- org.apache.curator:curator-framework:jar:2.6.0:compile
[INFO] | | +- org.apache.hadoop:hadoop-core:jar:2.5.0-mr1-cdh5.2.1:compile
[INFO] | | | +- javax.servlet.jsp:jsp-api:jar:2.1:compile
[INFO] | | | +- net.java.dev.jets3t:jets3t:jar:0.6.1:compile
[INFO] | | | +- hsqldb:hsqldb:jar:1.8.0.10:compile
[INFO] | | | \- org.eclipse.jdt:core:jar:3.1.1:compile
[INFO] | | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:2.5.0-cdh5.2.1:compile
[INFO] | | | \- commons-daemon:commons-daemon:jar:1.0.13:compile
[INFO] | | +- com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1:compile
[INFO] | | \- junit:junit:jar:4.11:compile
[INFO] | | \- org.hamcrest:hamcrest-core:jar:1.3:compile
[INFO] | +- com.jcraft:jsch:jar:0.1.51:compile
[INFO] | +- com.twitter.elephantbird:elephant-bird-core:jar:4.1:compile
[INFO] | | \- com.googlecode.json-simple:json-simple:jar:1.1:compile
[INFO] | \- com.twitter.elephantbird:elephant-bird-hadoop-compat:jar:4.1:compile
[INFO] +- com.typesafe:config:jar:1.2.1:compile
[INFO] +- org.json4s:json4s-jackson_2.10:jar:3.3.0.RC1:compile
[INFO] | \- org.json4s:json4s-core_2.10:jar:3.3.0.RC1:compile
[INFO] | +- org.json4s:json4s-ast_2.10:jar:3.3.0.RC1:compile
[INFO] | \- org.json4s:json4s-scalap_2.10:jar:3.3.0.RC1:compile
**[INFO] \- com.fasterxml.jackson.core:jackson-databind:jar:2.4.2:compile
[INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.4.0:compile
[INFO] \- com.fasterxml.jackson.core:jackson-core:jar:2.4.2:compile**
NoSuchMethodError 始终表示不兼容的 jar 文件。在你的情况下，我会说它在 jackson.module.scala 和 jackson.databind
请切换到两个文件的相同版本。
Hadoop 通常在类路径上有许多 jar 的自己的版本。您可以尝试通过将 mapreduce.job.user.classpath.first 设置为 true 来设置您的类路径以覆盖它。或者，您可以降级您的项目以使用与您的 hadoop 部署所使用的相同版本的 jackson。
非隔离类路径可能是 hadoop 最大的痛点。