处理 Python 请求中的错误

Question

我正在学习在 Python 中使用 requests，如果网站 根本不存在.

我查看了 this 问题，但不清楚问题的 OP 是否真的想检查该站点是否存在，或者只是 returns 一个错误。该问题的所有答案的问题在于，如果站点根本不存在，我们就无法真正使用 HTTP 响应 headers，因为没有从服务器返回任何响应不存在。

这是一个例子。

如果我使用此代码，我将不会收到任何错误，因为该站点已存在。

import requests
r = requests.get('https://duckduckgo.com')

但是，如果我输入一个我知道不存在的网页，我会得到一个错误

import requests
r = requests.get('https://thissitedoesnotexist.com')

if r.status_code == requests.codes.ok:
    print('Site good')
else:
    print('Site bad')

这个错误超长，如果网站不存在，我希望有一个更有意义、更短的错误。

Traceback (most recent call last):
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connection.py", line 416, in connect
    self.sock = ssl_wrap_socket(
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\ssl.py", line 512, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\ssl.py", line 1070, in _create
    self.do_handshake()
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\ssl.py", line 1341, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='234876.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:997)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\ADMIN\Desktop\tetst.py", line 2, in <module>
    r = requests.get('https://234876.com')
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='234876.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:997)')))

是否可以制作一个 returns 的函数，例如 print('The site probably does not exist') 或至少不会给出 EOF 错误？

Answer 1

做了一些更多的研究，刚刚了解到我需要使用 @Anand Sowmithiran 提到的 Python Try Except。这是一个为初学者解释的视频：https://www.youtube.com/watch?v=NIWwJbo-9_8

import requests

try:
    r = requests.get("http://www.duckduckgo.com")
except requests.exceptions.ConnectionError:
    print('\n\tSorry. There was a network problem getting the URL. Perhaps it does not exist?\n\tCheck the URL, DNS issues or if you are being rejected by the server.')
else:
    print(r)

Answer 2

通常最好的做法是捕获请求中的异常

您还可以在 Response 上使用 .raise_for_status() 以获得有意义的 Exception 非 OK 请求

但是，您要注意您要处理异常的地方

马上？您的程序可以有意义地处理它还是应该退出？
调用方应该处理特定的异常（例如 requests.exceptions.Timeout）还是更一般的异常？
你有很多函数相互调用吗？是否应该处理可能的 Exception 的某些子集？还有哪个？

请参阅Python Exception Hierarchy了解第一方异常继承结构

import sys
import requests

def some_function_which_makes_requests():
    r = requests.get("https://example.com", timeout=(2,10))
    r.raise_for_status()  # raise for non-OK
    return r.json()       # interpret response via some method (for example as JSON)

def main():
    ...
    try:
        result_json = some_function_which_makes_requests
    except requests.exceptions.Timeout:
        print("WARNING: request timed out")
        result_json = None  # still effectively handled for later program?
    except requests.exceptions.RequestException as ex:
        sys.exit(f"something wrong with Request: {repr(ex)}")
    except Exception:
        sys.exit(f"something wrong around Request: {repr(ex)}")
    # now you can use result_json

处理 Python 请求中的错误

Handeling errors in Python requests

python

http

web

screen-scraping