为什么 netstat 中的 Recv-Q 值等于 socket backlog + 1？

Question

当我执行netstat -tulnp时，输出如下：

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.11:43043        0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:8005          0.0.0.0:*               LISTEN      1/java
tcp        0      0 0.0.0.0:2021            0.0.0.0:*               LISTEN      1/java
tcp        0      0 0.0.0.0:22222           0.0.0.0:*               LISTEN      1/java
tcp        0      0 0.0.0.0:8719            0.0.0.0:*               LISTEN      1/java
tcp      101      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1/java
tcp       51      0 0.0.0.0:1234            0.0.0.0:*               LISTEN      1/java
tcp        0      0 0.0.0.0:20891           0.0.0.0:*               LISTEN      1/java
udp        0      0 127.0.0.11:55285        0.0.0.0:*                           -

Recv-Q 的值引起了我的注意。经过我的排查，发现是在JVM应用中发生了OOM，在日志中可以发现负责监听80端口的http-nio-80-Acceptor-0线程已经退出，负责调度1234端口的线程请求已退出。相关日志如下：

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "http-nio-80-Acceptor-0"
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Thread-5"

tomcat使用的默认配置，即backlog为100，源码位于tomcat/AbstractEndpoint.java at 8.5.59 · apache/tomcat · GitHub:

/**
  * Allows the server developer to specify the acceptCount (backlog) that
  * should be used for server sockets. By default, this value
  * is 100.
  */
private int acceptCount = 100;
public void setAcceptCount(int acceptCount) { if (acceptCount > 0) this.acceptCount = acceptCount; }
public int getAcceptCount() { return acceptCount; }

1234端口的监听由HTTPServer触发创建，创建代码为HttpServer.create(new InetSocketAddress(PROMETHEUS_SERVER_PORT), 0);，backlog在ServerSocket.java中修正为50，源码位于jdk/ServerSocket.java at jdk8-b120 · openjdk/jdk · GitHub:

public void bind(SocketAddress endpoint, int backlog) throws IOException {
    if (isClosed())
        throw new SocketException("Socket is closed");
    if (!oldImpl && isBound())
        throw new SocketException("Already bound");
    if (endpoint == null)
        endpoint = new InetSocketAddress(0);
    if (!(endpoint instanceof InetSocketAddress))
        throw new IllegalArgumentException("Unsupported address type");
    InetSocketAddress epoint = (InetSocketAddress) endpoint;
    if (epoint.isUnresolved())
        throw new SocketException("Unresolved address");
    if (backlog < 1)
      backlog = 50;
    try {
        SecurityManager security = System.getSecurityManager();
        if (security != null)
            security.checkListen(epoint.getPort());
        getImpl().bind(epoint.getAddress(), epoint.getPort());
        getImpl().listen(backlog);
        bound = true;
    } catch(SecurityException e) {
        bound = false;
        throw e;
    } catch(IOException e) {
        bound = false;
        throw e;
    }
}

从netstat(8) - Linux manual page我们知道，Recv-Q表示当socket处于Listening状态时当前syn backlog，令我困惑的是为什么Recv-Q是one 超过我们设定的 backlog 值？

Answer 1

TL;DR netstat/ss 正在报告完整和不完整的连接，而积压只关心已完成的连接

we know, Recv-Q indicates current syn backlog when socket is in Listening state,

是的，这在 sock_diag man page. It looks like netlink takes this value from the following 内核结构中得到证实：

rql.udiag_rqueue = sk->sk_receive_queue.qlen;

再看qlen，好像和backlog不一样。 qlen包括完整和不完整的连接，而backlog只关注完整的连接。 listen 手册页记录了这一点：

The behavior of the backlog argument on TCP sockets changed with Linux 2.2. Now it specifies the queue length for completely established sockets waiting to be accepted, instead of the number of incomplete connection requests.

至于为什么接收队列关心不完整的连接，我只能推测它正在尝试处理在 TCP 握手仍在发生时释放积压的边缘情况。

Answer 2

A backlog value of N really does mean allow "N + 1" connections to queue to a listening socket. This allows one to specify "0" as the backlog and still get 1 connection.

参考：NET: Revert incorrect accept queue backlog changes. · torvalds/linux@64a1465 · GitHub

为什么 netstat 中的 Recv-Q 值等于 socket backlog + 1？

Why is the Recv-Q value in netstat equal to socket backlog + 1?

sockets

networking

netstat

tcp

backlog