Storm执行后报错
Error After Storm Execution
我通过 DRPC 服务执行 Storm 拓扑。但是，我在下面看到一个错误。我不知道为什么我得到 IndexOutOfBoundsException。也许在 spout 中多次发射 ...
==============drpc spout源码（定制）=======================
_collector.emit(new Values("0:"+endNum*1/9, JSONValue.toJSONString(returnInfo)), new DRPCMessageId(req.get_request_id(), 0));
_collector.emit(new Values(Math.round(endNum*1.0/9.0)+":"+Math.round(endNum*2.0/9.0), JSONValue.toJSONString(returnInfo)), new DRPCMessageId(req.get_request_id(), 1));
_collector.emit(new Values(Math.round(endNum*2.0/9.0)+":"+Math.round(endNum*3.0/9.0), JSONValue.toJSONString(returnInfo)), new DRPCMessageId(req.get_request_id(), 2));
===================错误========================= ====
java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 3, Size: 1
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:87) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.disruptor$consume_batch.invoke(disruptor.clj:76) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$fn__6579$fn__6594$fn__6623.invoke(executor.clj:542) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__459.invoke(util.clj:463) ~[storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_11]
Caused by: java.lang.IndexOutOfBoundsException: Index: 3, Size: 1
at java.util.ArrayList.rangeCheck(ArrayList.java:638) ~[na:1.8.0_11]
at java.util.ArrayList.get(ArrayList.java:414) ~[na:1.8.0_11]
at com.skcc.drpctest.NewDRPCSpout.fail(NewDRPCSpout.java:198) ~[stormjar.jar:na]
at backtype.storm.daemon.executor$fail_spout_msg.invoke(executor.clj:372) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$fn$reify__6582.expire(executor.clj:432) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.utils.RotatingMap.rotate(RotatingMap.java:73) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$fn__6579$tuple_action_fn__6585.invoke(executor.clj:437) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$mk_task_receiver$fn__6570.invoke(executor.clj:404) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.disruptor$clojure_handler$reify__1605.onEvent(disruptor.clj:58) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) ~[storm-core-0.9.5.jar:0.9.5]
... 6 common frames omitted
2015-07-02T11:22:41.944+0900 b.s.d.executor [ERROR]
java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 3, Size: 1
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:87) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.disruptor$consume_batch.invoke(disruptor.clj:76) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$fn__6579$fn__6594$fn__6623.invoke(executor.clj:542) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__459.invoke(util.clj:463) ~[storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_11]
Caused by: java.lang.IndexOutOfBoundsException: Index: 3, Size: 1
at java.util.ArrayList.rangeCheck(ArrayList.java:638) ~[na:1.8.0_11]
at java.util.ArrayList.get(ArrayList.java:414) ~[na:1.8.0_11]
at com.skcc.drpctest.NewDRPCSpout.fail(NewDRPCSpout.java:198) ~[stormjar.jar:na]
at backtype.storm.daemon.executor$fail_spout_msg.invoke(executor.clj:372) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$fn$reify__6582.expire(executor.clj:432) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.utils.RotatingMap.rotate(RotatingMap.java:73) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$fn__6579$tuple_action_fn__6585.invoke(executor.clj:437) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$mk_task_receiver$fn__6570.invoke(executor.clj:404) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.disruptor$clojure_handler$reify__1605.onEvent(disruptor.clj:58) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) ~[storm-core-0.9.5.jar:0.9.5]
... 6 common frames omitted
2015-07-02T11:22:41.963+0900 b.s.util [ERROR] Halting process: ("Worker died")
java.lang.RuntimeException: ("Worker died")
at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:325) [storm-core-0.9.5.jar:0.9.5]
at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.5.1.jar:na]
at backtype.storm.daemon.worker$fn__7028$fn__7029.invoke(worker.clj:497) [storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$mk_executor_data$fn__6480$fn__6481.invoke(executor.clj:240) [storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__459.invoke(util.clj:473) [storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_11]
2015-07-02T11:22:41.965+0900 b.s.d.worker [INFO] Shutting down worker drpc-topology-5-1435802568 6f57193f-afa0-416d-9cae-d9b7fd66ff3d 54
2015-07-02T11:22:41.965+0900 b.s.m.n.Client [INFO] closing Netty Client Netty-Client-com.skcc.bigdata00/203.235.192.35:49
2015-07-02T11:22:41.965+0900 b.s.m.n.Client [INFO] waiting up to 600000 ms to send 0 pending messages to Netty-Client-com.skcc.bigdata00/203.235.192.35:49
2015-07-02T11:22:41.968+0900 b.s.m.n.Client [INFO] closing Netty Client Netty-Client-com.skcc.bigdata00/203.235.192.35:50
2015-07-02T11:22:41.969+0900 b.s.m.n.Client [INFO] waiting up to 600000 ms to send 0 pending messages to Netty-Client-com.skcc.bigdata00/203.235.192.35:50
2015-07-02T11:22:41.969+0900 b.s.d.worker [INFO] Shutting down receive thread
2015-07-02T11:22:41.970+0900 b.s.m.n.Client [INFO] creating Netty Client, connecting to localhost:54, bufferSize: 5242880
2015-07-02T11:22:41.970+0900 o.a.s.c.r.ExponentialBackoffRetry [WARN] maxRetries too large (300). Pinning to 29
2015-07-02T11:22:41.970+0900 b.s.u.StormBoundedExponentialBackoffRetry [INFO] The baseSleepTimeMs [100] the maxSleepTimeMs [1000] the maxRetries [300]
2015-07-02T11:22:41.974+0900 b.s.m.n.Client [INFO] connection attempt 1 to Netty-Client-localhost/127.0.0.1:54 scheduled to run in 0 ms
2015-07-02T11:22:41.975+0900 b.s.m.loader [INFO] Shutting down receiving-thread: [drpc-topology-5-1435802568, 54]
2015-07-02T11:22:41.975+0900 b.s.m.n.Client [ERROR] connection to Netty-Client-localhost/127.0.0.1:54 is unavailable
2015-07-02T11:22:41.975+0900 b.s.m.n.Client [ERROR] dropping 1 message(s) destined for Netty-Client-localhost/127.0.0.1:54
2015-07-02T11:22:41.975+0900 b.s.m.n.Client [INFO] closing Netty Client Netty-Client-localhost/127.0.0.1:54
2015-07-02T11:22:41.975+0900 b.s.m.n.Client [INFO] waiting up to 600000 ms to send 0 pending messages to Netty-Client-localhost/127.0.0.1:54
2015-07-02T11:22:41.975+0900 b.s.m.loader [INFO] Waiting for receiving-thread:[drpc-topology-5-1435802568, 54] to die
2015-07-02T11:22:41.978+0900 b.s.m.n.Client [INFO] connection established to Netty-Client-localhost/127.0.0.1:54
2015-07-02T11:22:41.978+0900 b.s.m.loader [INFO] Shutdown receiving-thread: [drpc-topology-5-1435802568, 54]
2015-07-02T11:22:41.978+0900 b.s.d.worker [INFO] Shut down receive thread
2015-07-02T11:22:41.978+0900 b.s.d.worker [INFO] Terminating messaging context
2015-07-02T11:22:41.978+0900 b.s.d.worker [INFO] Shutting down executors
2015-07-02T11:22:41.979+0900 b.s.d.executor [INFO] Shutting down executor drpc:[4 4]
2015-07-02T11:22:41.980+0900 b.s.util [INFO] Async loop interrupted!
堆栈跟踪表明您有一个 IndexOutOfBoundsException。因此，很可能您正试图从 Spout 中的错误数组索引中读取 (NewDRPCSpout.java)。
我通过 DRPC 服务执行 Storm 拓扑。但是，我在下面看到一个错误。我不知道为什么我得到 IndexOutOfBoundsException。也许在 spout 中多次发射 ...
==============drpc spout源码（定制）=======================
_collector.emit(new Values("0:"+endNum*1/9, JSONValue.toJSONString(returnInfo)), new DRPCMessageId(req.get_request_id(), 0));
_collector.emit(new Values(Math.round(endNum*1.0/9.0)+":"+Math.round(endNum*2.0/9.0), JSONValue.toJSONString(returnInfo)), new DRPCMessageId(req.get_request_id(), 1));
_collector.emit(new Values(Math.round(endNum*2.0/9.0)+":"+Math.round(endNum*3.0/9.0), JSONValue.toJSONString(returnInfo)), new DRPCMessageId(req.get_request_id(), 2));
===================错误========================= ====
java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 3, Size: 1
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:87) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.disruptor$consume_batch.invoke(disruptor.clj:76) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$fn__6579$fn__6594$fn__6623.invoke(executor.clj:542) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__459.invoke(util.clj:463) ~[storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_11]
Caused by: java.lang.IndexOutOfBoundsException: Index: 3, Size: 1
at java.util.ArrayList.rangeCheck(ArrayList.java:638) ~[na:1.8.0_11]
at java.util.ArrayList.get(ArrayList.java:414) ~[na:1.8.0_11]
at com.skcc.drpctest.NewDRPCSpout.fail(NewDRPCSpout.java:198) ~[stormjar.jar:na]
at backtype.storm.daemon.executor$fail_spout_msg.invoke(executor.clj:372) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$fn$reify__6582.expire(executor.clj:432) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.utils.RotatingMap.rotate(RotatingMap.java:73) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$fn__6579$tuple_action_fn__6585.invoke(executor.clj:437) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$mk_task_receiver$fn__6570.invoke(executor.clj:404) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.disruptor$clojure_handler$reify__1605.onEvent(disruptor.clj:58) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) ~[storm-core-0.9.5.jar:0.9.5]
... 6 common frames omitted
2015-07-02T11:22:41.944+0900 b.s.d.executor [ERROR]
java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 3, Size: 1
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:87) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.disruptor$consume_batch.invoke(disruptor.clj:76) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$fn__6579$fn__6594$fn__6623.invoke(executor.clj:542) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__459.invoke(util.clj:463) ~[storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_11]
Caused by: java.lang.IndexOutOfBoundsException: Index: 3, Size: 1
at java.util.ArrayList.rangeCheck(ArrayList.java:638) ~[na:1.8.0_11]
at java.util.ArrayList.get(ArrayList.java:414) ~[na:1.8.0_11]
at com.skcc.drpctest.NewDRPCSpout.fail(NewDRPCSpout.java:198) ~[stormjar.jar:na]
at backtype.storm.daemon.executor$fail_spout_msg.invoke(executor.clj:372) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$fn$reify__6582.expire(executor.clj:432) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.utils.RotatingMap.rotate(RotatingMap.java:73) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$fn__6579$tuple_action_fn__6585.invoke(executor.clj:437) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$mk_task_receiver$fn__6570.invoke(executor.clj:404) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.disruptor$clojure_handler$reify__1605.onEvent(disruptor.clj:58) ~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) ~[storm-core-0.9.5.jar:0.9.5]
... 6 common frames omitted
2015-07-02T11:22:41.963+0900 b.s.util [ERROR] Halting process: ("Worker died")
java.lang.RuntimeException: ("Worker died")
at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:325) [storm-core-0.9.5.jar:0.9.5]
at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.5.1.jar:na]
at backtype.storm.daemon.worker$fn__7028$fn__7029.invoke(worker.clj:497) [storm-core-0.9.5.jar:0.9.5]
at backtype.storm.daemon.executor$mk_executor_data$fn__6480$fn__6481.invoke(executor.clj:240) [storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__459.invoke(util.clj:473) [storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_11]
2015-07-02T11:22:41.965+0900 b.s.d.worker [INFO] Shutting down worker drpc-topology-5-1435802568 6f57193f-afa0-416d-9cae-d9b7fd66ff3d 54
2015-07-02T11:22:41.965+0900 b.s.m.n.Client [INFO] closing Netty Client Netty-Client-com.skcc.bigdata00/203.235.192.35:49
2015-07-02T11:22:41.965+0900 b.s.m.n.Client [INFO] waiting up to 600000 ms to send 0 pending messages to Netty-Client-com.skcc.bigdata00/203.235.192.35:49
2015-07-02T11:22:41.968+0900 b.s.m.n.Client [INFO] closing Netty Client Netty-Client-com.skcc.bigdata00/203.235.192.35:50
2015-07-02T11:22:41.969+0900 b.s.m.n.Client [INFO] waiting up to 600000 ms to send 0 pending messages to Netty-Client-com.skcc.bigdata00/203.235.192.35:50
2015-07-02T11:22:41.969+0900 b.s.d.worker [INFO] Shutting down receive thread
2015-07-02T11:22:41.970+0900 b.s.m.n.Client [INFO] creating Netty Client, connecting to localhost:54, bufferSize: 5242880
2015-07-02T11:22:41.970+0900 o.a.s.c.r.ExponentialBackoffRetry [WARN] maxRetries too large (300). Pinning to 29
2015-07-02T11:22:41.970+0900 b.s.u.StormBoundedExponentialBackoffRetry [INFO] The baseSleepTimeMs [100] the maxSleepTimeMs [1000] the maxRetries [300]
2015-07-02T11:22:41.974+0900 b.s.m.n.Client [INFO] connection attempt 1 to Netty-Client-localhost/127.0.0.1:54 scheduled to run in 0 ms
2015-07-02T11:22:41.975+0900 b.s.m.loader [INFO] Shutting down receiving-thread: [drpc-topology-5-1435802568, 54]
2015-07-02T11:22:41.975+0900 b.s.m.n.Client [ERROR] connection to Netty-Client-localhost/127.0.0.1:54 is unavailable
2015-07-02T11:22:41.975+0900 b.s.m.n.Client [ERROR] dropping 1 message(s) destined for Netty-Client-localhost/127.0.0.1:54
2015-07-02T11:22:41.975+0900 b.s.m.n.Client [INFO] closing Netty Client Netty-Client-localhost/127.0.0.1:54
2015-07-02T11:22:41.975+0900 b.s.m.n.Client [INFO] waiting up to 600000 ms to send 0 pending messages to Netty-Client-localhost/127.0.0.1:54
2015-07-02T11:22:41.975+0900 b.s.m.loader [INFO] Waiting for receiving-thread:[drpc-topology-5-1435802568, 54] to die
2015-07-02T11:22:41.978+0900 b.s.m.n.Client [INFO] connection established to Netty-Client-localhost/127.0.0.1:54
2015-07-02T11:22:41.978+0900 b.s.m.loader [INFO] Shutdown receiving-thread: [drpc-topology-5-1435802568, 54]
2015-07-02T11:22:41.978+0900 b.s.d.worker [INFO] Shut down receive thread
2015-07-02T11:22:41.978+0900 b.s.d.worker [INFO] Terminating messaging context
2015-07-02T11:22:41.978+0900 b.s.d.worker [INFO] Shutting down executors
2015-07-02T11:22:41.979+0900 b.s.d.executor [INFO] Shutting down executor drpc:[4 4]
2015-07-02T11:22:41.980+0900 b.s.util [INFO] Async loop interrupted!
堆栈跟踪表明您有一个 IndexOutOfBoundsException。因此，很可能您正试图从 Spout 中的错误数组索引中读取 (NewDRPCSpout.java)。