将 google 云语音 api 与 tornado 服务器一起使用时多个 CLOSE_WAIT。打开的文件太多错误

Question

我在 CLOSE_WAIT 中获得同一进程的多个线程，因此我收到 'too many files open' 错误。

OSError: [Errno 24] Too many open files:

多次调用 google 云语音 api 时会发生这种情况。

已经在 Whosebug 上浏览了各种答案，但我无法找出解决方案。

sudo lsof | grep -i close | wc -l

15180

我分享的代码是实际代码的精简版。我可以使用下面的代码重现错误。

import os
import tornado.httpserver, tornado.ioloop, tornado.options, tornado.web, tornado.escape
import os.path
import string
import json
from google.cloud import speech
from google.cloud.speech import types, enums

tornado.options.parse_command_line()
tornado.options.define("port", default=8888, help="run on the given port", type=int)
SPEECH_TO_TEXT_CREDENTIALS = 'my_json_file.json'
UPLOAD_FOLDER = '/home/ubuntu/uploads'

class Application(tornado.web.Application):

    def __init__(self):
        os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = SPEECH_TO_TEXT_CREDENTIALS
        self.speech_client = speech.SpeechClient()
        handlers = [
            (r"/test_bug/client/googlestt2", GoogleSTTHandler)
        ]
        tornado.web.Application.__init__(self, handlers)

class GoogleSTTHandler(tornado.web.RequestHandler):
    def post(self):
        if 'audio' not in self.request.files:
            self.finish({'Error': "No audio provided"})
        audio_filename =  'test.wav'
        audio = self.request.files['audio'][0]
        with open(os.path.join(UPLOAD_FOLDER, audio_filename), 'wb') as f:
            f.write(audio['body'])
        with open(os.path.join(UPLOAD_FOLDER, audio_filename), 'rb') as audio_file:
            content = audio_file.read()
        audio = types.RecognitionAudio(content=content)
        config = types.RecognitionConfig(encoding=enums.RecognitionConfig.AudioEncoding.LINEAR16, language_code='en-IN')
        response = self.application.speech_client.recognize(config, audio)
        if not response.results:
            Transcript_Upload = "Empty Audio"
        else:
            for result in response.results:
                Transcript_Upload = 'Transcript: {}'.format(result.alternatives[0].transcript)
        self.finish(Transcript_Upload)

def main():
    http_server = tornado.httpserver.HTTPServer(Application())
    http_server.listen(tornado.options.options.port)
    tornado.ioloop.IOLoop.instance().start()

if __name__ == "__main__":
   main()

如果我做错了什么请提出建议以及如何解决这个问题。

Answer 1

google-cloud-python 和 gcloud-python - https://github.com/googleapis/google-cloud-python/issues/5570 中的这个已知问题。我放弃了它，从那以后我一直在直接使用 google API。

作为旁注，您正在使用同步 API，但要利用 Tornado（实际上是任何异步框架），您应该使用异步 libs/calls 等 google-cloud-python's Asynchronous Recognition

将 google 云语音 api 与 tornado 服务器一起使用时多个 CLOSE_WAIT。打开的文件太多错误

Multiple CLOSE_WAIT when using google cloud speech api with tornado server. Too many open files error

python

linux

tornado

google-cloud-speech