Redis, Jedis 连接池优化和监控工具

Question

我是Redis新手，Jedis。今天，我在日志中看到一个错误

Caused by: redis.clients.jedis.exceptions.JedisExhaustedPoolException: redis.clients.jedis.exceptions.JedisExhaustedPoolException: Could not get a resource since the pool is exhausted
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_191]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_191]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_191]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_191]
        at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:593) ~[?:1.8.0_191]
        at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) ~[?:1.8.0_191]
        at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) ~[?:1.8.0_191]
        at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) ~[?:1.8.0_191]
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174) ~[?:1.8.0_191]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) ~[?:1.8.0_191]
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) ~[?:1.8.0_191]
        at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583) ~[?:1.8.0_191]
        at com.nokia.snmpapp.VariableDataUtils.processSnmpDataByOP(VariableDataUtils.java:372) ~[snmpapp.jar:?]
       at com.nokia.snmpapp.consumerservice.DocsisQos3MibData.processDocsQos3ParamSetTableObjectEvent(DocsisQos3MibData.java:172) ~[snmpapp.jar:?]
        at com.nokia.snmpapp.KafkaConsumer.onSnmpappDocsQos3ParamSetTableObjectEvent(KafkaConsumer.java:315) ~[snmpapp.jar:?]
        ... 16 more 

Caused by: redis.clients.jedis.exceptions.JedisExhaustedPoolException: Could not get a resource since the pool is exhausted
            at redis.clients.jedis.util.Pool.getResource(Pool.java:53) ~[snmpapp.jar:?]
            at redis.clients.jedis.JedisPool.getResource(JedisPool.java:234) ~[snmpapp.jar:?]
            at com.nokia.snmpapp.RedisUtils.add2list(RedisUtils.java:27) ~[snmpapp.jar:?]
            at com.nokia.snmpapp.VariableDataUtils.lambda$processSnmpDataByOP(VariableDataUtils.java:379) ~[snmpapp.jar:?]
            at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) ~[?:1.8.0_191]
            at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[?:1.8.0_191]
            at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_191]
            at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291) ~[?:1.8.0_191]
            at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731) ~[?:1.8.0_191]
            at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) ~[?:1.8.0_191]
            at java.util.concurrent.ForkJoinPool$WorkQueue.execLocalTasks(ForkJoinPool.java:1040) ~[?:1.8.0_191]
            at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1058) ~[?:1.8.0_191]
            at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) ~[?:1.8.0_191]
            at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) ~[?:1.8.0_191]  

Caused by: java.util.NoSuchElementException: Timeout waiting for idle object               
           at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:439) ~[snmpapp.jar:?]
           at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:349) ~[snmpapp.jar:?]
           at redis.clients.jedis.util.Pool.getResource(Pool.java:50) ~[snmpapp.jar:?]
           at redis.clients.jedis.JedisPool.getResource(JedisPool.java:234) ~[snmpapp.jar:?]
           at com.nokia.snmpapp.RedisUtils.add2list(RedisUtils.java:27) ~[snmpapp.jar:?]
           at com.nokia.snmpapp.VariableDataUtils.lambda$processSnmpDataByOP(VariableDataUtils.java:379) ~[snmpapp.jar:?]
           at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) ~[?:1.8.0_191]
           at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[?:1.8.0_191]
           at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_191]
           at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291) ~[?:1.8.0_191]
           at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731) ~[?:1.8.0_191]
           at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) ~[?:1.8.0_191]
           at java.util.concurrent.ForkJoinPool$WorkQueue.execLocalTasks(ForkJoinPool.java:1040) ~[?:1.8.0_191]
           at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1058) ~[?:1.8.0_191]
           at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) ~[?:1.8.0_191]
           at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) ~[?:1.8.0_191]

我目前的配置是

  maxTotal: 10
  maxIdle: 5
  minIdle: 1
  maxWaitMillis: 2000

我知道我可以增加连接池大小来解决错误消息。但是，在为我的环境确定一个合适的数字之前，我想多了解一下资源使用情况。我已经 google 讨论过这个问题，但我找不到太多关于监控工具或如何监控性能和池使用情况的话题。这是否意味着没有工具可以监控？如果没有办法监控连接池的使用情况，我该怎么做才能更好地了解要优化 Redis 连接的内容？

提前致谢

Answer 1

使用 JMX MBean 监控 Jedis 池

Jedis Pool 基于 Apache Generic-Pool API，因此您可以使用 JMX 监控连接池的状态。

例如，在启用 JMX 的情况下启动您的应用程序：

-Dcom.sun.management.jmxremote.port=9999 
-Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.ssl=false

然后连接 JConsole 或任何其他 JMX 兼容工具：

连接到您的 application/process
转到 MBean
转到 org.apache.commons.pool2.GenericObjectPool.pool MBean

您将能够监控您的 application/JedisPool。

发生了什么

您有许多符合 JMX 的监控工具，允许您根据阈值发出警报

大小调整和配置

你的池的配置确实是特定于应用程序的，要配置 maxTotal 属性，那就是你必须查看的最大连接数：

您的服务需要多少并发连接
每次调用执行您发送的命令需要多长时间

并且您需要确保您的系统（客户端和服务器端）不会过载。

JedisPool：一定要释放连接；）

另外，这不是关于监控，而是关于最佳实践：

您的应用程序会在每次使用后释放与池的连接吗？

/// Jedis implements Closeable. Hence, the jedis instance will be auto-closed after the last statement.
try (Jedis jedis = pool.getResource()) {

   // use your jedis
  jedis.set("foo", "bar");

} // the resource will be released back to the pool

如果不对资源使用 try，您必须关闭连接（将其放回池中）

Jedis jedis = null;
try {
  jedis = pool.getResource();
  // work with Redis
  jedis.set("foo", "bar");
} finally {
  // you must close the connection to put it back to the pool
  if (jedis != null) {
    jedis.close();
  }
}

还有当您关闭应用程序时：

 pool.close()

Redis, Jedis 连接池优化和监控工具

Redis, Jedis connection pool optimizeand monitoring tools

redis

jedis