ClassNotFoundException 而运行 Kafka hadoop-consumer

Question

我是 Kafka 的新手，我正在 hortonworks sanbox 2.3 上进行测试。我正在关注 kafka 教程 https://github.com/kafka-dev/kafka/tree/master/contrib/hadoop-consumer.

当运行宁./运行-class.sh kafka.etl.impl.DataGenerator test/test.properties，我收到错误

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2615)
        at java.lang.Class.getMethod0(Class.java:2856)
        at java.lang.Class.getMethod(Class.java:1668)
        at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
        at  sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
        at java.net.URLClassLoader.run(URLClassLoader.java:366)
        at java.net.URLClassLoader.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        ... 6 more

我已经在运行-class.sh 中包含了所有可能的 kafka jar，但仍然没有成功。请帮忙！

Answer 1

您缺少 class org.apache.hadoop.conf.Configuration 的 jar，不是 Kafka 库的一部分。它应该包含在 hadoop-common-2.5.0 中。您必须将此 jar 添加到 class 路径。

ClassNotFoundException 而运行 Kafka hadoop-consumer

ClassNotFoundException while running Kafka hadoop-consumer

hadoop-streaming

kafka-consumer-api

ClassNotFoundException 而 运行 Kafka hadoop-consumer

ClassNotFoundException while running Kafka hadoop-consumer

hadoop-streaming

kafka-consumer-api

ClassNotFoundException 而运行 Kafka hadoop-consumer