接收数据导致 "too many files open"

Question

在我的客户端中，我通过 ZeroMQ 收到大量需要不断更新的输入。我的服务器是用 python 编写的，但这应该无关紧要。这就是我在 MainActivity:

中所做的

public class MainActivity extends AppCompatActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        /********************************NETWORK********************************/
        new NetworkCall().execute("");
    }

    private class NetworkCall extends AsyncTask<String, Void, String> {

        @Override
        protected String doInBackground(String... params) {
            while (true) {
                try {
                    ZMQ.Context context = ZMQ.context(1);

                    // Connect to server
                    ZMQ.Socket requester = context.socket(ZMQ.REQ);
                    String address = "tcp://xxx.xx.xx.xx";
                    int port = 5000;
                    requester.connect(address + ":" + port);

                    // Initialize poll set
                    ZMQ.Poller poller = new ZMQ.Poller(1);
                    poller.register(requester, ZMQ.Poller.POLLIN);

                    requester.send("COORDINATES");

                    //while (true) {
                    String data;
                    poller.poll();

                    data = requester.recvStr();
                    System.out.println(data);

                    if (data == null) {
                        try {
                            sleep(100);
                        } catch (InterruptedException e) {
                            e.printStackTrace();
                        }
                    } requester.close();
                } catch (IllegalStateException ise) {
                    ise.printStackTrace();
                }
            }
        }

        @Override
        protected void onPostExecute(String result) {
        }

        @Override
        protected void onPreExecute() {
        }

        @Override
        protected void onProgressUpdate(Void... values) {
        }
    }
}

在我的设备上执行此代码后，我会从服务器收到大约 5-9 个输入数据字符串，但随后出现以下异常：

E/AndroidRuntime: FATAL EXCEPTION: AsyncTask #2
                  Process: com.example.viktoria.gazefocus, PID: 31339
                  java.lang.RuntimeException: An error occurred while executing doInBackground()
                      at android.os.AsyncTask.done(AsyncTask.java:353)
                      at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:383)
                      at java.util.concurrent.FutureTask.setException(FutureTask.java:252)
                      at java.util.concurrent.FutureTask.run(FutureTask.java:271)
                      at android.os.AsyncTask$SerialExecutor.run(AsyncTask.java:245)
                      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
                      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
                      at java.lang.Thread.run(Thread.java:764)
                   Caused by: com.example.viktoria.gazefocus.zmq.ZError$IOException: java.io.IOException: Too many open files
                      at com.example.viktoria.gazefocus.zmq.Signaler.makeFdPair(Signaler.java:94)
                      at com.example.viktoria.gazefocus.zmq.Signaler.<init>(Signaler.java:50)
                      at com.example.viktoria.gazefocus.zmq.Mailbox.<init>(Mailbox.java:51)
                      at com.example.viktoria.gazefocus.zmq.Ctx.<init>(Ctx.java:128)
                      at com.example.viktoria.gazefocus.zmq.ZMQ.zmq_ctx_new(ZMQ.java:244)
                      at com.example.viktoria.gazefocus.zmq.ZMQ.zmqInit(ZMQ.java:277)
                      at org.zeromq.ZMQ$Context.<init>(ZMQ.java:269)
                      at org.zeromq.ZMQ.context(ZMQ.java:254)
                      at com.example.viktoria.gazefocus.MainActivity$NetworkCall.doInBackground(MainActivity.java:73)
                      at com.example.viktoria.gazefocus.MainActivity$NetworkCall.doInBackground(MainActivity.java:67)
                      at android.os.AsyncTask.call(AsyncTask.java:333)
                      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                      at android.os.AsyncTask$SerialExecutor.run(AsyncTask.java:245) 
                      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162) 
                      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636) 
                      at java.lang.Thread.run(Thread.java:764) 
                   Caused by: java.io.IOException: Too many open files
                      at sun.nio.ch.IOUtil.makePipe(Native Method)
                      at sun.nio.ch.PipeImpl.<init>(PipeImpl.java:42)
                      at sun.nio.ch.SelectorProviderImpl.openPipe(SelectorProviderImpl.java:50)
                      at java.nio.channels.Pipe.open(Pipe.java:155)
                      at com.example.viktoria.gazefocus.zmq.Signaler.makeFdPair(Signaler.java:91)
                      at com.example.viktoria.gazefocus.zmq.Signaler.<init>(Signaler.java:50) 
                      at com.example.viktoria.gazefocus.zmq.Mailbox.<init>(Mailbox.java:51) 
                      at com.example.viktoria.gazefocus.zmq.Ctx.<init>(Ctx.java:128) 
                      at com.example.viktoria.gazefocus.zmq.ZMQ.zmq_ctx_new(ZMQ.java:244) 
                      at com.example.viktoria.gazefocus.zmq.ZMQ.zmqInit(ZMQ.java:277) 
                      at org.zeromq.ZMQ$Context.<init>(ZMQ.java:269) 
                      at org.zeromq.ZMQ.context(ZMQ.java:254) 
                      at com.example.viktoria.gazefocus.MainActivity$NetworkCall.doInBackground(MainActivity.java:73) 
                      at com.example.viktoria.gazefocus.MainActivity$NetworkCall.doInBackground(MainActivity.java:67) 
                      at android.os.AsyncTask.call(AsyncTask.java:333) 
                      at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
                      at android.os.AsyncTask$SerialExecutor.run(AsyncTask.java:245) 
                      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162) 
                      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636) 
                      at java.lang.Thread.run(Thread.java:764)

显然打开的文件过多。经过研究（我使用的是 Ubuntu 16.04），我将 ulimit 更改为 ulimit -n 10000。仍然会发生此异常。有时我得到更多的输入数据，有时更少。另外，如果我在 onCreate() 方法中设置类似 Executor executor = Executors.newFixedThreadPool(5); 的内容，则不会有任何改变。

如何克服这个问题？

感谢阅读！

Answer 1

你有泄漏，因为你没有关闭/结束/释放一些东西。我认为必须终止上下文：context.term() 在您关闭请求者之后...

Answer 2

嗯，在 distributed-system 设计中，用于信令/消息传递的基础设施设置成本不可忽略。有些 use-cases 更宽容，有些则更少。

总是为每个 method-call 获取一个新的 Context() 实例，并在 clean-up 调用它的 [= =29=].term()*-方法肯定比 hung-app 或冻结设备更好，但它远非公平设计，尊重进程延迟和 "ecology"-of-resources.

最好首先设置一个 semi-persistent 资源基础设施（每个 Context()-实例通常是一个非常昂贵的实例化玩具（API 4.2+ as of 2018-Q1），而不是Socket()-实例如此尖锐，但类似于 Poller() 和所有它的 internal-AccessPoint(s) registration-hooks，但该原理也可以扩展到它们上）。 =17=]

代码的早期 re-factoring 将有助于避免使用昂贵的资源作为 "consumable disposable"。

栏目：

      while (true) {
             try {
                    ZMQ.Context context = ZMQ.context(1);

                    //                     Connect to server
                    ZMQ.Socket requester = context.socket( ZMQ.REQ );
                    String address = "tcp://xxx.xx.xx.xx";
                    int port = 5000;
                    requester.connect( address + ":" + port );
                    ...
             }
             ...
      }

完全是 resources-devastating anti-pattern，伴随着 remote-hangups 和 remote-rejections 以及类似问题的重复延迟甚至风险。

接收数据导致 "too many files open"

Receiving data causes "too many files open"

java

android

networking

zeromq