com.datastax.driver.core.exceptions.BusyPoolException

Question

每当我在 Cassandra 的 table 中插入超过 1000 条数据并通过 id 获取数据时，它会抛出以下异常：

com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.BusyPoolException: [localhost/127.0.0.1] Pool is busy (no available connection and the queue has reached its max size 256)))
    at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:213)
    at com.datastax.driver.core.RequestHandler.access00(RequestHandler.java:49)
    at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:277)
    at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onFailure(RequestHandler.java:340)
    at com.google.common.util.concurrent.Futures.run(Futures.java:1764)
    at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456)
    at com.google.common.util.concurrent.Futures$ImmediateFuture.addListener(Futures.java:153)
    at com.google.common.util.concurrent.Futures.addCallback(Futures.java:1776)
    at com.google.common.util.concurrent.Futures.addCallback(Futures.java:1713)
    at com.datastax.driver.core.RequestHandler$SpeculativeExecution.query(RequestHandler.java:299)
    at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:274)
    at com.datastax.driver.core.RequestHandler.startNewExecution(RequestHandler.java:117)
    at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:97)
    at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:132)
    at com.outworkers.phantom.builder.query.CassandraOperations$class.scalaQueryStringToPromise(CassandraOperations.scala:67)
    at com.outworkers.phantom.builder.query.InsertQuery.scalaQueryStringToPromise(InsertQuery.scala:31)
    at com.outworkers.phantom.builder.query.CassandraOperations$class.scalaQueryStringExecuteToFuture(CassandraOperations.scala:31)
    at com.outworkers.phantom.builder.query.InsertQuery.scalaQueryStringExecuteToFuture(InsertQuery.scala:31)
    at com.outworkers.phantom.builder.query.ExecutableStatement$class.future(ExecutableQuery.scala:80)
    at com.outworkers.phantom.builder.query.InsertQuery.future(InsertQuery.scala:31)
    at nd.cluster.data.store.Points.upsert(Models.scala:114)

我已经使用 PoolingOptions 解决了上述问题。

val poolingOptions = new PoolingOptions()
    .setConnectionsPerHost(HostDistance.LOCAL, 1, 200)
    .setMaxRequestsPerConnection(HostDistance.LOCAL, 256)
    .setNewConnectionThreshold(HostDistance.LOCAL, 100).setCoreConnectionsPerHost(HostDistance.LOCAL, 200)

  val builder1 =  ContactPoint.local
    .noHeartbeat()
    .withClusterBuilder(_.withoutJMXReporting()
      .withoutMetrics().withPoolingOptions(poolingOptions)).keySpace("nd")

现在它甚至可以处理 1l 数据。但我不确定它的效率。谁能帮帮我？

Answer 1

这意味着您提交了太多的请求，并且没有等到期货完成后再提交更多。

每个连接的默认最大请求数是 1024。如果所有连接都超过了这个数字，连接池会将一些请求排入队列，最多 256 个。如果队列已满，将抛出 BusyPoolException。当然，您可以增加每个连接的最大请求数，以及每个主机的最大连接数。但真正的解决方案当然是限制你的线程。你可以例如以 1,000 个为一批提交您的请求，然后等待 futures 完成再提交更多，或者使用信号量来调节未决请求的总数并确保它们不超过一定数量（理论上，这个数字必须保持below num_hosts * max_connections_per_host * max_requests_per_connection – 实际上，我不建议超过 1,000，因为它可能不会给你带来更多的吞吐量）。

您可能会发现此链接很有用。

https://github.com/redisson/redisson/issues/438
https://groups.google.com/a/lists.datastax.com/forum/#!topic/java-driver-user/p3CwOL0kNrs http://docs.datastax.com/en/developer/java-driver/3.1/manual/pooling

com.datastax.driver.core.exceptions.BusyPoolException

com.datastax.driver.core.exceptions.BusyPoolException

cassandra

datastax