google.inject 不匹配的 HBase MapReduce
HBase MapReduce with google.inject mismatch
我正在尝试使用我们使用 Guice 4.0
的内部框架针对 HBase table 进行映射缩减
看起来 yarn 可能使用的是 3.0 版，我发现类型不匹配。
2016-03-25 08:19:45,784 ERROR [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.NoSuchMethodError: com.google.inject.multibindings.MultibindingsScanner.binder()Lcom/shaded/google/inject/Binder;
at com.google.inject.multibindings.MultibindingsScanner.configure(MultibindingsScanner.java:53)
at com.google.inject.AbstractModule.configure(AbstractModule.java:59)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:223)
at com.google.inject.AbstractModule.install(AbstractModule.java:118)
at my.packagename.client.KafkaModule.configure(KafkaModule.java:36)
at com.google.inject.AbstractModule.configure(AbstractModule.java:59)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:223)
at com.google.inject.spi.Elements.getElements(Elements.java:101)
at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:133)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:103)
at com.google.inject.Guice.createInjector(Guice.java:95)
at com.google.inject.Guice.createInjector(Guice.java:72)
at com.google.inject.Guice.createInjector(Guice.java:62)
at my.packagename.hbase_fix.CleanerMapper.setup(CleanerMapper.java:82)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
在同一份工作中，我与 fasterxml 有类似的不匹配，所以使用 maven-shade-plugin 成功地重新定位，但我没有用 com.google.inject
成功
<relocations>
<relocation>
<pattern>com.fasterxml</pattern>
<shadedPattern>com.shaded.fasterxml</shadedPattern>
<excludes>
<exclude>com.fasterxml.*</exclude>
</excludes>
</relocation>
<relocation>
<pattern>org.apache.commons.lang</pattern>
<shadedPattern>org.shaded.apache.commons.lang</shadedPattern>
<excludes>
<exclude>org.apache.commons.lang.*</exclude>
</excludes>
</relocation>
<relocation>
<pattern>com.google.inject.Binder</pattern>
<shadedPattern>com.shaded.google.inject.Binder</shadedPattern>
</relocation>
<relocation>
<pattern>com.google.inject.multibindings</pattern>
<shadedPattern>com.shaded.google.inject.multibindings</shadedPattern>
</relocation>
</relocations>
关于如何在 MapReduce 作业中特别是 google.inject 获得回合类型不匹配的任何建议？
我认为对此没有任何简单的解决方案。最后归结为强制 mapreduce 类路径先有 guice4 的副本，这样代码就会 运行.
不理想，但找到了唯一的解决方案
我正在尝试使用我们使用 Guice 4.0
的内部框架针对 HBase table 进行映射缩减看起来 yarn 可能使用的是 3.0 版，我发现类型不匹配。
2016-03-25 08:19:45,784 ERROR [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.NoSuchMethodError: com.google.inject.multibindings.MultibindingsScanner.binder()Lcom/shaded/google/inject/Binder;
at com.google.inject.multibindings.MultibindingsScanner.configure(MultibindingsScanner.java:53)
at com.google.inject.AbstractModule.configure(AbstractModule.java:59)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:223)
at com.google.inject.AbstractModule.install(AbstractModule.java:118)
at my.packagename.client.KafkaModule.configure(KafkaModule.java:36)
at com.google.inject.AbstractModule.configure(AbstractModule.java:59)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:223)
at com.google.inject.spi.Elements.getElements(Elements.java:101)
at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:133)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:103)
at com.google.inject.Guice.createInjector(Guice.java:95)
at com.google.inject.Guice.createInjector(Guice.java:72)
at com.google.inject.Guice.createInjector(Guice.java:62)
at my.packagename.hbase_fix.CleanerMapper.setup(CleanerMapper.java:82)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
在同一份工作中，我与 fasterxml 有类似的不匹配，所以使用 maven-shade-plugin 成功地重新定位，但我没有用 com.google.inject
<relocations>
<relocation>
<pattern>com.fasterxml</pattern>
<shadedPattern>com.shaded.fasterxml</shadedPattern>
<excludes>
<exclude>com.fasterxml.*</exclude>
</excludes>
</relocation>
<relocation>
<pattern>org.apache.commons.lang</pattern>
<shadedPattern>org.shaded.apache.commons.lang</shadedPattern>
<excludes>
<exclude>org.apache.commons.lang.*</exclude>
</excludes>
</relocation>
<relocation>
<pattern>com.google.inject.Binder</pattern>
<shadedPattern>com.shaded.google.inject.Binder</shadedPattern>
</relocation>
<relocation>
<pattern>com.google.inject.multibindings</pattern>
<shadedPattern>com.shaded.google.inject.multibindings</shadedPattern>
</relocation>
</relocations>
关于如何在 MapReduce 作业中特别是 google.inject 获得回合类型不匹配的任何建议？
我认为对此没有任何简单的解决方案。最后归结为强制 mapreduce 类路径先有 guice4 的副本，这样代码就会 运行.
不理想，但找到了唯一的解决方案