Storm 2.0.0 内存不足
Storm 2.0.0 going Out Of Memory
我将我的代码库从 1.1.1 升级为使用 storm 2.0.0。现在我观察到如果我在本地模式下 运行 拓扑，它会在几分钟后内存不足。
[THREAD ID=AsyncLocalizer Executor - 2-EventThread] Dev-APC180-local o.a.s.s.o.a.z.ClientCnxn Error while calling watcher java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at org.apache.storm.shade.org.apache.zookeeper.ClientCnxn.start(ClientCnxn.java:421) at org.apache.storm.shade.org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:454) at org.apache.storm.shade.org.apache.curator.utils.DefaultZookeeperFactory.newZooKeeper(DefaultZookeeperFactory.java:29) at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.newZooKeeper(CuratorFrameworkImpl.java:213) at org.apache.storm.shade.org.apache.curator.HandleHolder.getZooKeeper(HandleHolder.java:101) at org.apache.storm.shade.org.apache.curator.HandleHolder.getZooKeeper(HandleHolder.java:57) at org.apache.storm.shade.org.apache.curator.ConnectionState.reset(ConnectionState.java:204) at org.apache.storm.shade.org.apache.curator.ConnectionState.handleExpiredSession(ConnectionState.java:380) at org.apache.storm.shade.org.apache.curator.ConnectionState.checkState(ConnectionState.java:315) at org.apache.storm.shade.org.apache.curator.ConnectionState.process(ConnectionState.java:169) at org.apache.storm.shade.org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:533) at org.apache.storm.shade.org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:508)
以上是错误的堆栈跟踪。在更深入的分析中，我发现 'AsyncLocalizer EventThread' 和 'AsyncLocalizer SendThread' 大约有 5000 个线程。它由 AsyncLocalizer.updateBlobs.
生成
AsyncLocalizer.updateBlobs 是一个计划任务，每 30 秒 运行s。请指出正确的方向。我对我错过的东西一无所知。
这很可能是由于 https://issues.apache.org/jira/browse/STORM-3501。 Blob 清理在 2.0.0 中被破坏，因此主管不断尝试下载实际删除的 blob。我认为这也会导致它启动大量的 Curator 实例。
我将我的代码库从 1.1.1 升级为使用 storm 2.0.0。现在我观察到如果我在本地模式下 运行 拓扑，它会在几分钟后内存不足。
[THREAD ID=AsyncLocalizer Executor - 2-EventThread] Dev-APC180-local o.a.s.s.o.a.z.ClientCnxn Error while calling watcher java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at org.apache.storm.shade.org.apache.zookeeper.ClientCnxn.start(ClientCnxn.java:421) at org.apache.storm.shade.org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:454) at org.apache.storm.shade.org.apache.curator.utils.DefaultZookeeperFactory.newZooKeeper(DefaultZookeeperFactory.java:29) at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.newZooKeeper(CuratorFrameworkImpl.java:213) at org.apache.storm.shade.org.apache.curator.HandleHolder.getZooKeeper(HandleHolder.java:101) at org.apache.storm.shade.org.apache.curator.HandleHolder.getZooKeeper(HandleHolder.java:57) at org.apache.storm.shade.org.apache.curator.ConnectionState.reset(ConnectionState.java:204) at org.apache.storm.shade.org.apache.curator.ConnectionState.handleExpiredSession(ConnectionState.java:380) at org.apache.storm.shade.org.apache.curator.ConnectionState.checkState(ConnectionState.java:315) at org.apache.storm.shade.org.apache.curator.ConnectionState.process(ConnectionState.java:169) at org.apache.storm.shade.org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:533) at org.apache.storm.shade.org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:508)
以上是错误的堆栈跟踪。在更深入的分析中，我发现 'AsyncLocalizer EventThread' 和 'AsyncLocalizer SendThread' 大约有 5000 个线程。它由 AsyncLocalizer.updateBlobs.
生成AsyncLocalizer.updateBlobs 是一个计划任务，每 30 秒 运行s。请指出正确的方向。我对我错过的东西一无所知。
这很可能是由于 https://issues.apache.org/jira/browse/STORM-3501。 Blob 清理在 2.0.0 中被破坏，因此主管不断尝试下载实际删除的 blob。我认为这也会导致它启动大量的 Curator 实例。