特威皮 OpenSSL.SSL.WantReadError

Tweepy OpenSSL.SSL.WantReadError

Python 3.6。我使用 tweepy 的流媒体来获取推文。它运作良好。但有时,如果我让它打开超过 24 小时,就会出现此错误

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\contrib\pyopenssl.py", line 277, in recv_into
return self.connection.recv_into(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1547, in recv_into
self._raise_ssl_error(self._ssl, result)
    File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1353, in _raise_ssl_error
raise WantReadError()
  OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:


Traceback (most recent call last):
 File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\contrib\pyopenssl.py", line 277, in recv_into
return self.connection.recv_into(*args, **kwargs)
    File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1547, in recv_into
self._raise_ssl_error(self._ssl, result)
  File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1370, in _raise_ssl_error
raise SysCallError(errno, errorcode.get(errno))
OpenSSL.SSL.SysCallError: (10054, 'WSAECONNRESET')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\response.py", line 302, in _error_catcher
yield
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\response.py", line 384, in read
data = self._fp.read(amt)
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 449, in read
n = self.readinto(b)
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 483, in readinto
return self._readinto_chunked(b)
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 578, in _readinto_chunked
chunk_left = self._get_chunk_left()
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 546, in _get_chunk_left
chunk_left = self._read_next_chunk_size()
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 506, in _read_next_chunk_size
line = self.fp.readline(_MAXLINE + 1)
  File "C:\ProgramData\Anaconda3\lib\socket.py", line 586, in readinto
return self._sock.recv_into(b)
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\contrib\pyopenssl.py", line 293, in recv_into
return self.recv_into(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\contrib\pyopenssl.py", line 282, in recv_into
raise SocketError(str(e))
OSError: (10054, 'WSAECONNRESET')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\threading.py", line 916, in _bootstrap_inner
self.run()
  File "C:\ProgramData\Anaconda3\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
 File "twitter_aspi_v0.8.py", line 179, in _init_stream
tweepy.Stream(auth, listener).userstream()
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 396, in userstream
self._start(async)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 363, in _start
self._run()
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 296, in _run
raise exception
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 265, in _run
self._read_loop(resp)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 315, in _read_loop
line = buf.read_line().strip()
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 180, in read_line
self._buffer += self._stream.read(self._chunk_size)
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\response.py", line 401, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "C:\ProgramData\Anaconda3\lib\contextlib.py", line 100, in __exit__
self.gen.throw(type, value, traceback)
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\response.py", line 320, in _error_catcher
raise ProtocolError('Connection broken: %r' % e, e)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection broken: OSError("(10054, \'WSAECONNRESET\')",)', OSError("(10054, 'WSAECONNRESET')",))

我的代码很长,关于错误,它似乎来自 urllib3、OpenSSL 和 tweepy 访问 Twitter 的方式 API。所以我可以在启动流媒体之前用 try 来处理这个问题,但我想知道是否有更好的修复方法可以理解和避免这种情况?谢谢!

根据 Twitter 开发人员文档:rate limiting当您超过使用限制时,预计会连接 reset/failure。

Requests / 15-min window (user auth) = 900

Requests / 15-min window (app auth) = 1500

下面也写的很清楚

如果最初的重新连接尝试不成功,您的客户端应使用指数 back-off 模式继续尝试重新连接,直到成功重新连接。

(更新)

Regardless of how your client gets disconnected, you should configure your app to reconnect immediately. If your first reconnection attempt is unsuccessful, we recommend that your app implement an exponential back-off pattern in subsequent reconnection attempts (e.g. wait 1 second, then 2 seconds, then 4, 8, 16, etc), with some reasonable upper limit. If this upper limit is reached, you should configure your client to notify your team so that you can investigate further.

标准(免费)Twitter API 即 Tweepy API 由 REST API 和 Streaming API 组成。 Streaming API 提供对推文的 low-latency 访问。当列入白名单时,广告 API 有其他限制。

休息API限制

Clients may access a theoretical maximum of 3,200 statuses via the page and count parameters for the user_timeline REST API methods. Other timeline methods have a theoretical maximum of 800 statuses. Requests for more than the limit will result in a reply with a status code of 200 and an empty result in the format requested. Twitter still maintains a database of all the Tweets sent by a user. However, to ensure performance, this limit is in place on the API calls.

这可能是用户不应该强制执行垃圾邮件的简单原因。

解决方案:

您可能会 exception 和 re-establish 连接到 Twitter 并继续阅读推文。

不幸的是,到目前为止,除了从 Twitter 获得更好的使用津贴之外别无选择。

这看起来更像是 Tweepy 未处理的临时连接超时,因此您应该围绕它编写一个包装器并捕获异常并重新启动它。我认为无法避免异常,因为您连接到外部站点,有时它可能会超时

你应该看看这个 http://docs.tweepy.org/en/v3.5.0/streaming_how_to.html#handling-errors 的错误处理部分,看看 on_error 在你的情况下是否在连接超时发生时被调用

class MyStreamListener(tweepy.StreamListener):

    def on_error(self, status_code):
        if status_code == 420:
            #returning False in on_data disconnects the stream
            return False

如果这没有帮助,则使用包装器方法