Apache Storm 未 运行 正确并且 worker.log 生成异常

Apache Storm is not running properly and worker.log generating exceptions

我正在使用 Apache Storm 和 Apache Kafka。使用的 spout 是 kafka-spout,但它没有从 Kafka 主题中读取任何内容。我不知道这是否与我在风暴工人中面临的问题有关。如果您知道任何相关解决方案,请告诉我并通知我。

Storm 拓扑已成功上传到 Storm。我获取了日志,这是工作人员在他们的 worker.log 文件中写的内容:

java.lang.ExceptionInInitializerError
    at java.nio.file.FileSystems.getDefault(FileSystems.java:176) ~[?:1.8.0_91]
    at java.nio.file.Paths.get(Paths.java:84) ~[?:1.8.0_91]
    at org.apache.storm.util$is_absolute_path_QMARK_.invoke(util.clj:68) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.config$absolute_storm_local_dir.invoke(config.clj:94) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.config$worker_root.invoke(config.clj:290) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.config$worker_root.invoke(config.clj:292) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.config$worker_pids_root.invoke(config.clj:296) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.config$worker_pid_path.invoke(config.clj:304) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto____8451.invoke(worker.clj:592) ~[storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.applyToHelper(AFn.java:178) ~[clojure-1.7.0.jar:?]
    at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
    at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$fn__8450$mk_worker__8545.doInvoke(worker.clj:583) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$_main.invoke(worker.clj:771) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.applyToHelper(AFn.java:165) [clojure-1.7.0.jar:?]
    at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker.main(Unknown Source) [storm-core-1.0.1.jar:1.0.1]

Caused by: java.security.PrivilegedActionException
    at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:96) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.<clinit>(FileSystems.java:90) ~[?:1.8.0_91]
    ... 18 more

Caused by: sun.nio.fs.UnixException: No such file or directory
    at sun.nio.fs.UnixNativeDispatcher.getcwd(Native Method) ~[?:1.8.0_91]
    at sun.nio.fs.UnixFileSystem.<init>(UnixFileSystem.java:67) ~[?:1.8.0_91]
    at sun.nio.fs.LinuxFileSystem.<init>(LinuxFileSystem.java:39) ~[?:1.8.0_91]
    at sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:46) ~[?:1.8.0_91]
    at sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:39) ~[?:1.8.0_91]
    at sun.nio.fs.UnixFileSystemProvider.<init>(UnixFileSystemProvider.java:56) ~[?:1.8.0_91]
    at sun.nio.fs.LinuxFileSystemProvider.<init>(LinuxFileSystemProvider.java:41) ~[?:1.8.0_91]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_91]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_91]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_91]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_91]
    at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_91]
    at sun.nio.fs.DefaultFileSystemProvider.createProvider(DefaultFileSystemProvider.java:48) ~[?:1.8.0_91]
    at sun.nio.fs.DefaultFileSystemProvider.create(DefaultFileSystemProvider.java:63) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(FileSystems.java:108) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.access[=12=]0(FileSystems.java:89) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.run(FileSystems.java:98) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.run(FileSystems.java:96) ~[?:1.8.0_91]
    at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:96) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.<clinit>(FileSystems.java:90) ~[?:1.8.0_91]
    ... 18 more

2016-07-24 13:14:33.154 o.a.s.util [ERROR] Halting process: ("Error on initialization")
java.lang.RuntimeException: ("Error on initialization")
    at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$fn__8450$mk_worker__8545.doInvoke(worker.clj:583) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$_main.invoke(worker.clj:771) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.applyToHelper(AFn.java:165) [clojure-1.7.0.jar:?]
    at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker.main(Unknown Source) [storm-core-1.0.1.jar:1.0.1]

解决办法是检查Storm本地集群的任务并行数。在他们的最新版本中,他们为此任务添加了最大并行数 property.That 是自动删除额外工作人员的文件夹。只是把worker的数量从3个改成2个或更少,才能正常执行。

int workers = 2;

conf.setMaxTaskParallelism(workers);

感谢大家的评论。