Hazelcast 内存不足问题

Hazelcast Out of memory issue

Hazelcast 3.3 中经常出现 内存不足 问题,我想知道节点是否有足够的内存和 CPU,16GB Ram,8 核处理器,仍然发生此错误。它是 单节点 设置。

com.hazelcast.core.HazelcastException: Client is shutting down!!!
    at com.hazelcast.client.connection.nio.ClientConnection.innerClose(ClientConnection.java:302)
    at com.hazelcast.client.connection.nio.ClientConnection.close(ClientConnection.java:355)
    at com.hazelcast.client.connection.nio.ClientConnection.close(ClientConnection.java:230)
    at com.hazelcast.client.connection.nio.ClientConnectionManagerImpl.shutdown(ClientConnectionManagerImpl.java:249)
    at com.hazelcast.client.ClientOutOfMemoryHandler$ClientHelper.closeSockets(ClientOutOfMemoryHandler.java:54)
    at com.hazelcast.client.ClientOutOfMemoryHandler$ClientHelper.cleanResources(ClientOutOfMemoryHandler.java:45)
    at com.hazelcast.client.ClientOutOfMemoryHandler.onOutOfMemory(ClientOutOfMemoryHandler.java:34)
    at com.hazelcast.instance.OutOfMemoryErrorDispatcher.onOutOfMemory(OutOfMemoryErrorDispatcher.java:143)
    at com.hazelcast.spi.impl.BasicOperationService$OperationHandler.handleOperationError(BasicOperationService.java:827)
    at com.hazelcast.spi.impl.BasicOperationService$OperationHandler.handle(BasicOperationService.java:703)
    at com.hazelcast.spi.impl.BasicOperationService$OperationHandler.access0(BasicOperationService.java:675)
    at com.hazelcast.spi.impl.BasicOperationService.runOperationOnCallingThread(BasicOperationService.java:229)
    at com.hazelcast.spi.impl.BasicInvocation.doInvokeLocal(BasicInvocation.java:286)
    at com.hazelcast.spi.impl.BasicInvocation.doInvoke(BasicInvocation.java:272)
    at com.hazelcast.spi.impl.BasicInvocation.invoke(BasicInvocation.java:232)
    at com.hazelcast.spi.impl.BasicInvocationBuilder.invoke(BasicInvocationBuilder.java:45)
    at com.hazelcast.client.impl.client.PartitionClientRequest.process(PartitionClientRequest.java:60)
    at com.hazelcast.client.impl.ClientEngineImpl$ClientPacketProcessor.processRequest(ClientEngineImpl.java:418)
    at com.hazelcast.client.impl.ClientEngineImpl$ClientPacketProcessor.run(ClientEngineImpl.java:357)
    at com.hazelcast.spi.impl.BasicOperationService$BasicDispatcherImpl.dispatch(BasicOperationService.java:546)
    at com.hazelcast.spi.impl.BasicOperationScheduler$OperationThread.process(BasicOperationScheduler.java:439)
    at com.hazelcast.spi.impl.BasicOperationScheduler$OperationThread.doRun(BasicOperationScheduler.java:433)
    at com.hazelcast.spi.impl.BasicOperationScheduler$OperationThread.run(BasicOperationScheduler.java:408)
    at ------ End remote and begin local stack-trace ------.(Unknown Source)
    at com.hazelcast.client.spi.impl.ClientCallFuture.resolveResponse(ClientCallFuture.java:192)
    at com.hazelcast.client.spi.impl.ClientCallFuture.get(ClientCallFuture.java:134)
    at com.hazelcast.client.spi.impl.ClientCallFuture.get(ClientCallFuture.java:111)
    at com.hazelcast.client.spi.ClientProxy.invoke(ClientProxy.java:110)
    at com.hazelcast.client.proxy.ClientMapProxy.get(ClientMapProxy.java:180)

对于每个新 运行,这发生在 运行.

(大约)15 分钟之后

在此之后,我们不断收到以下错误,

java.util.concurrent.RejectedExecutionException: Internal executor rejected task: java.util.concurrent.FutureTask@5463e258, because client is shutting down...
    at com.hazelcast.client.spi.impl.ClientExecutionServiceImpl.rejectedExecution(ClientExecutionServiceImpl.java:60)
    at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
    at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
    at com.hazelcast.client.spi.impl.ClientExecutionServiceImpl.submitInternal(ClientExecutionServiceImpl.java:81)
    at com.hazelcast.client.connection.nio.ClientConnectionManagerImpl.getOrConnect(ClientConnectionManagerImpl.java:339)
    at com.hazelcast.client.connection.nio.ClientConnectionManagerImpl.tryToConnect(ClientConnectionManagerImpl.java:280)
    at com.hazelcast.client.spi.impl.ClientInvocationServiceImpl.send(ClientInvocationServiceImpl.java:120)
    at com.hazelcast.client.spi.impl.ClientInvocationServiceImpl.invokeOnRandomTarget(ClientInvocationServiceImpl.java:61)
    at com.hazelcast.client.spi.ProxyManager.initialize(ProxyManager.java:197)
    at com.hazelcast.client.spi.ProxyManager.getOrCreateProxy(ProxyManager.java:180)
    at com.hazelcast.client.HazelcastClient.getDistributedObject(HazelcastClient.java:461)
    at com.hazelcast.client.HazelcastClient.getMap(HazelcastClient.java:314)
    at com.hazelcast.client.HazelcastClientProxy.getMap(HazelcastClientProxy.java:97)

请问有没有设置我需要添加的???

对于信息:

没有添加驱逐政策,默认为NONE。 我正在使用 MAP STORE 写入延迟为 1 秒的持久性策略。

由于您没有支持合同,这是最有价值的答案。

  1. 你有一个单节点设置,单节点集群听起来是错误的,运行 Hazelcast 也是错误的。 Hazelcast 不是为单个成员集群设计的(它也没有意义)。

  2. 堆大小设置为16GB。您期望的数据值是多少?我的意思是你不能在 16GB 的堆中存储 16GB 的数据。

  3. 您是否尝试过在 JVM 运行 OOME 时获取堆转储?您是否调查过它,看看是什么在消耗您的记忆? (如前所述,寻找 "how to find memory leaks")

  4. 如果您没有选择驱逐,即使使用 MapStore,您也不会被驱逐。

  5. 您是否尝试过不同的 GC 组合或尝试查看 GC?有不同类型的 GC 错误。

  6. 您的客户端似乎 运行ning OOME 所以问题可能根本不是数据量。但是由于没有进一步的信息,所以很难推理。