在 MongoDB 中打开连接的 SocketTimeout
SocketTimeout with opened connection in MongoDB
我有一个 Java 应用程序在 MongoDB 上执行一些聚合，但有时它只是挂起并抛出 SocketTimeout 异常。在出现异常后，应用程序将 运行 正常（稍等片刻，然后它可能会再次引发异常）。
我刚刚发现 this explanation 这似乎是一个可能的原因，但我不确定。
我初始化 Mongo客户端并保持与数据库的连接打开。我不确定这是否是个问题，我应该每次都获取数据库，然后让数据库被垃圾收集（并关闭连接）。
另一种方法可以是定期 ping Mongo 以保持连接池 "fresh"。
使用的客户端是这样的：
public class DbClient {
private static MongoClient mongoClient;
private static MongoDatabase db;
private DbClient() {}
public static void init() throws Exception {
mongoClient = new MongoClient();
}
public static MongoDatabase getDB() {
if(mongoClient == null)
throw new IllegalStateException("Client not initialized!");
if(db == null) {
db = mongoClient.getDatabase("my_db");
}
return db;
}
}
这是 SocketTimeout 的可能原因吗？
这是抛出的异常：
09:20:45.742 [qtp605535417-46] INFO org.mongodb.driver.connection - Closed connection [connectionId{localValue:16, serverValue:6562}] to myapp.com:27017 because there was a socket exception raised by this connection.
09:20:45.743 [qtp605535417-46] ERROR myapp.service.Api - Error processing request
com.mongodb.MongoSocketReadTimeoutException: Timeout while receiving message
at com.mongodb.connection.InternalStreamConnection.translateReadException(InternalStreamConnection.java:474) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:225) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.UsageTrackingInternalConnection.receiveMessage(UsageTrackingInternalConnection.java:102) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.DefaultConnectionPool$PooledConnection.receiveMessage(DefaultConnectionPool.java:435) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:112) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:159) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:286) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:173) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:215) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:206) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:112) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.FindOperation.call(FindOperation.java:487) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.FindOperation.call(FindOperation.java:482) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:239) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:212) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.FindOperation.execute(FindOperation.java:482) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.FindOperation.execute(FindOperation.java:79) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.Mongo.execute(Mongo.java:772) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.Mongo.execute(Mongo.java:759) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.OperationIterable.iterator(OperationIterable.java:47) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.FindIterableImpl.iterator(FindIterableImpl.java:143) ~[mongo-java-driver-3.2.2.jar!/:na]
at myapp.common.db.service.dao.AnalysisMongoImpl.getAnalysis(AnalysisMongoImpl.java:66) ~[common-0.2.0-SNAPSHOT.jar!/:na]
at myapp.common.db.service.AnalysisServiceImpl.getAnalysis(AnalysisServiceImpl.java:31) ~[common-0.2.0-SNAPSHOT.jar!/:na]
at myapp.aggregator.service.Api.handle(Api.java:88) ~[aggregator-0.2.0-SNAPSHOT.jar!/:na]
at spark.webserver.MatcherFilter.doFilter(MatcherFilter.java:139) [spark-core-1.1.1.jar!/:na]
at spark.webserver.JettyHandler.doHandle(JettyHandler.java:54) [spark-core-1.1.1.jar!/:na]
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:179) [jetty-server-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136) [jetty-server-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.server.Server.handle(Server.java:451) [jetty-server-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:252) [jetty-server-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:266) [jetty-server-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:240) [jetty-io-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:596) [jetty-util-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.util.thread.QueuedThreadPool.run(QueuedThreadPool.java:527) [jetty-util-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_95]
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.7.0_95]
at java.net.SocketInputStream.read(SocketInputStream.java:152) ~[na:1.7.0_95]
at java.net.SocketInputStream.read(SocketInputStream.java:122) ~[na:1.7.0_95]
at com.mongodb.connection.SocketStream.read(SocketStream.java:85) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:491) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:221) ~[mongo-java-driver-3.2.2.jar!/:na]
... 34 common frames omitted
经过几次尝试，我发现这是 Azure's Load Balancer 的问题。
闲置 60 秒后，它将断开任何挂起的 TCP 连接。
进一步挖掘后，我发现了 MongoDB diagnostics FAQ 的 post，并且我将 tcp keepalive 设置为 120s：
sudo sysctl -w net.ipv4.tcp_keepalive_time=<value>
我还将 MongoClient 的 socketKeepAlive 设置为 true:
MongoClientOptions.Builder options = MongoClientOptions.builder();
options.socketKeepAlive(true);
mongoClient = new MongoClient(mongoAddress, options.build());
修复这些问题后，问题似乎消失了！
如果您在客户端和服务器之间或分片集群或副本集的成员之间遇到没有其他合理原因的套接字错误，请检查 TCP keepalive 值（例如，tcp_keepalive_time 值 Linux 系统）。一个常见的keepalive周期是7200秒（2小时）；但是，不同的发行版和 macOS 可能有不同的设置。
对于 MongoDB，保持活动时间较短（大约 120 秒（两分钟））会获得更好的结果。
在您安装 mongodb 的位置，您只需 运行 在 Linux
上执行此命令
sudo sysctl -w net.ipv4.tcp_keepalive_time=120
我刚遇到同样的问题，通过在 Spring 启动配置中增加 spring.data.mongodb.socketTimeout=<value> 解决了这个问题。默认情况下，该值为 0，表示不超时。但是，我的同事将它设置为 20000（20 秒），所以一旦我的 aggregate() 运行时间超过 20 秒，MongoSocketReadTimeoutException 就会被丢弃！
我有一个 Java 应用程序在 MongoDB 上执行一些聚合，但有时它只是挂起并抛出 SocketTimeout 异常。在出现异常后，应用程序将 运行 正常（稍等片刻，然后它可能会再次引发异常）。
我刚刚发现 this explanation 这似乎是一个可能的原因，但我不确定。
我初始化 Mongo客户端并保持与数据库的连接打开。我不确定这是否是个问题，我应该每次都获取数据库，然后让数据库被垃圾收集（并关闭连接）。
另一种方法可以是定期 ping Mongo 以保持连接池 "fresh"。
使用的客户端是这样的：
public class DbClient {
private static MongoClient mongoClient;
private static MongoDatabase db;
private DbClient() {}
public static void init() throws Exception {
mongoClient = new MongoClient();
}
public static MongoDatabase getDB() {
if(mongoClient == null)
throw new IllegalStateException("Client not initialized!");
if(db == null) {
db = mongoClient.getDatabase("my_db");
}
return db;
}
}
这是 SocketTimeout 的可能原因吗？
这是抛出的异常：
09:20:45.742 [qtp605535417-46] INFO org.mongodb.driver.connection - Closed connection [connectionId{localValue:16, serverValue:6562}] to myapp.com:27017 because there was a socket exception raised by this connection.
09:20:45.743 [qtp605535417-46] ERROR myapp.service.Api - Error processing request
com.mongodb.MongoSocketReadTimeoutException: Timeout while receiving message
at com.mongodb.connection.InternalStreamConnection.translateReadException(InternalStreamConnection.java:474) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:225) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.UsageTrackingInternalConnection.receiveMessage(UsageTrackingInternalConnection.java:102) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.DefaultConnectionPool$PooledConnection.receiveMessage(DefaultConnectionPool.java:435) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:112) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:159) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:286) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:173) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:215) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:206) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:112) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.FindOperation.call(FindOperation.java:487) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.FindOperation.call(FindOperation.java:482) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:239) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:212) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.FindOperation.execute(FindOperation.java:482) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.operation.FindOperation.execute(FindOperation.java:79) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.Mongo.execute(Mongo.java:772) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.Mongo.execute(Mongo.java:759) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.OperationIterable.iterator(OperationIterable.java:47) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.FindIterableImpl.iterator(FindIterableImpl.java:143) ~[mongo-java-driver-3.2.2.jar!/:na]
at myapp.common.db.service.dao.AnalysisMongoImpl.getAnalysis(AnalysisMongoImpl.java:66) ~[common-0.2.0-SNAPSHOT.jar!/:na]
at myapp.common.db.service.AnalysisServiceImpl.getAnalysis(AnalysisServiceImpl.java:31) ~[common-0.2.0-SNAPSHOT.jar!/:na]
at myapp.aggregator.service.Api.handle(Api.java:88) ~[aggregator-0.2.0-SNAPSHOT.jar!/:na]
at spark.webserver.MatcherFilter.doFilter(MatcherFilter.java:139) [spark-core-1.1.1.jar!/:na]
at spark.webserver.JettyHandler.doHandle(JettyHandler.java:54) [spark-core-1.1.1.jar!/:na]
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:179) [jetty-server-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136) [jetty-server-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.server.Server.handle(Server.java:451) [jetty-server-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:252) [jetty-server-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:266) [jetty-server-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:240) [jetty-io-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:596) [jetty-util-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at org.eclipse.jetty.util.thread.QueuedThreadPool.run(QueuedThreadPool.java:527) [jetty-util-9.0.2.v20130417.jar!/:9.0.2.v20130417]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_95]
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.7.0_95]
at java.net.SocketInputStream.read(SocketInputStream.java:152) ~[na:1.7.0_95]
at java.net.SocketInputStream.read(SocketInputStream.java:122) ~[na:1.7.0_95]
at com.mongodb.connection.SocketStream.read(SocketStream.java:85) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:491) ~[mongo-java-driver-3.2.2.jar!/:na]
at com.mongodb.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:221) ~[mongo-java-driver-3.2.2.jar!/:na]
... 34 common frames omitted
经过几次尝试，我发现这是 Azure's Load Balancer 的问题。
闲置 60 秒后，它将断开任何挂起的 TCP 连接。
进一步挖掘后，我发现了 MongoDB diagnostics FAQ 的 post，并且我将 tcp keepalive 设置为 120s：
sudo sysctl -w net.ipv4.tcp_keepalive_time=<value>
我还将 MongoClient 的 socketKeepAlive 设置为 true:
MongoClientOptions.Builder options = MongoClientOptions.builder();
options.socketKeepAlive(true);
mongoClient = new MongoClient(mongoAddress, options.build());
修复这些问题后，问题似乎消失了！
如果您在客户端和服务器之间或分片集群或副本集的成员之间遇到没有其他合理原因的套接字错误，请检查 TCP keepalive 值（例如，tcp_keepalive_time 值 Linux 系统）。一个常见的keepalive周期是7200秒（2小时）；但是，不同的发行版和 macOS 可能有不同的设置。
对于 MongoDB，保持活动时间较短（大约 120 秒（两分钟））会获得更好的结果。
在您安装 mongodb 的位置，您只需 运行 在 Linux
上执行此命令sudo sysctl -w net.ipv4.tcp_keepalive_time=120
我刚遇到同样的问题，通过在 Spring 启动配置中增加 spring.data.mongodb.socketTimeout=<value> 解决了这个问题。默认情况下，该值为 0，表示不超时。但是，我的同事将它设置为 20000（20 秒），所以一旦我的 aggregate() 运行时间超过 20 秒，MongoSocketReadTimeoutException 就会被丢弃！