"Connection reset" Mac 上的 Apache HttpClient 4.5 异常

Question

我最多需要 1000 个线程才能同时发送 HTTP 请求。然而，在 Mac 上，我似乎达到了一些限制，这导致 Apache HTTPClient 出现 return 异常：

INFO: I/O exception (java.net.SocketException) caught when processing request to {}->http://my.url:80: Connection reset

限制似乎介于 200 到 300 个并发连接之间。

我创建了一个简单的应用程序，即使使用该应用程序也能够重现该问题：

import java.io.IOException;

import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;

public class App implements Runnable
{
    private static final int NUMBER_THREADS = 1000;

    public static void main( String[] args )
    {
        Thread[] threads = new Thread[NUMBER_THREADS];
        for(int i = 0; i < NUMBER_THREADS; i++)
        {
            threads[i] = new Thread(new App()); 
        }
        for(int i = 0; i < NUMBER_THREADS; i++)
        {
            threads[i].start(); 
        }
    }

    public void run() 
    {
        CloseableHttpClient httpclient = HttpClients.createDefault();
        HttpGet httpget = new HttpGet("http://my.url/test.html");
        CloseableHttpResponse response = null;
        try 
        {
            System.out.println( System.nanoTime() + " " + Thread.currentThread().getName() + " started" );
            response = httpclient.execute(httpget);
            System.out.println( System.nanoTime() + " " + Thread.currentThread().getName() + " finished" );
        } 
        catch (ClientProtocolException e) 
        {
            e.printStackTrace();
        } 
        catch (IOException e) 
        {
            e.printStackTrace();
        } 
        finally 
        {
            try 
            {
                if(response != null)
                {
                    response.close();
                }
            } 
            catch (IOException e) 
            {
                e.printStackTrace();
            }
        }
    }
}

我更新了一些 Mac 设置，但这没有任何帮助：

> sysctl -a | grep files
kern.maxfiles = 12288
kern.maxfilesperproc = 10240
kern.maxfiles: 12288
kern.maxfilesperproc: 10240
kern.num_files: 3202
> ulimit -n 10240
> ulimit -n
10240
> sysctl -a | grep somax
kern.ipc.somaxconn: 128
> sudo sysctl -w kern.ipc.somaxconn=2048
Password:
kern.ipc.somaxconn: 128 -> 2048
> sysctl -a | grep somax
kern.ipc.somaxconn: 2048

所以我想了解这个限制是从哪里来的？ OsX / Apache HttpClient 本身？是否可以控制它？

提前致谢。

Answer 1

问题确实出在最大文件上，但我没有正确更改它们。 This 文章通过展示在 Mac OS X Yosemite / El Capitan 上更新文件限制的正确程序帮助我解决了这个问题。

创建和更新 /Library/LaunchDaemons/limit.maxfiles.plist 文件：

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
    <dict>
      <key>Label</key>
        <string>limit.maxfiles</string>
      <key>ProgramArguments</key>
        <array>
          <string>launchctl</string>
          <string>limit</string>
          <string>maxfiles</string>
          <string>65536</string>
          <string>65536</string>
        </array>
      <key>RunAtLoad</key>
        <true/>
      <key>ServiceIPC</key>
        <false/>
    </dict>
  </plist>

创建 /Library/LaunchDaemons/limit.maxproc.plist 文件：

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple/DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
    <dict>
      <key>Label</key>
        <string>limit.maxproc</string>
      <key>ProgramArguments</key>
        <array>
          <string>launchctl</string>
          <string>limit</string>
          <string>maxproc</string>
          <string>2048</string>
          <string>2048</string>
        </array>
      <key>RunAtLoad</key>
        <true />
      <key>ServiceIPC</key>
        <false />
    </dict>

使用以下行添加 .bashprofile:
```
ulimit -n 65536
ulimit -u 2048
```
重新启动系统

"Connection reset" Mac 上的 Apache HttpClient 4.5 异常

"Connection reset" exception with Apache HttpClient 4.5 on Mac

java

apache

macos

httpclient

apache-httpclient-4.x