HTTP header cut in half with `urllib3.exceptions.HeaderParsingError: [MissingHeaderBodySeparatorDefect()], unparsed data`

Question

我在日志中发现了一个奇怪的警告：

[WARNING] urllib3.connectionpool:467: Failed to parse headers (url=https://REDACTED): [MissingHeaderBodySeparatorDefect()], unparsed data: 'trol,Content-Type\r\n\r\n'
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 465, in _make_request
assert_header_parsing(httplib_response.msg)
File "/usr/local/lib/python3.8/dist-packages/urllib3/util/response.py", line 91, in assert_header_parsing
raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
urllib3.exceptions.HeaderParsingError: [MissingHeaderBodySeparatorDefect()], unparsed data: 'trol,Content-Type\r\n\r\n'

这是通过在我完全控制的 Web 服务（nginx 后面的 Python 应用程序）上调用标准 requests.post()。

当我在 http.client.HTTPResponse 中打开 debuglevel=1 时，我看到了这个：

reply: 'HTTP/1.1 200 OK\r\n'
header: Server: nginx/1.18.0 (Ubuntu)
header: Date: Tue, 30 Nov 2021 22:14:04 GMT
header: Content-Type: application/json
header: Transfer-Encoding: chunked
header: Connection: keep-alive
header: Vary: Accept-Encoding
header: Access-Control-Allow-Origin: *
header: Access-Control-Allow-Credentials: true
header: Access-Control-Allow-Methods: GET, POST, OPTIONS
header: Access-Control-Allow-Headers: DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Con

注意最后一个 header 在 ,If-Modified-Since,Cache-Con 中突然结束。

显然，requests==2.26.0（通过 urllib3==1.26.7 通过 http.client）在解析过程中出于某种原因将最后一个 header 切成两半，然后抱怨它“离开了over”数据与剩余的 trol,Content-Type\r\n\r\n.

在这种情况下警告并不重要，因为 header 并不是真正需要的。但令人恐惧的是，这种情况正在发生，因为……还有什么被删减/错误解析了？

同一个端点可以正常工作，例如curl:

$ curl -i -XPOST https://REDACTED
HTTP/1.1 200 OK
Server: nginx/1.18.0 (Ubuntu)
Date: Sat, 04 Dec 2021 20:08:59 GMT
Content-Type: application/json
Content-Length: 53
Connection: keep-alive
Vary: Accept-Encoding
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Headers: DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Con
trol,Content-Type

…JSON response…

知道哪里出了问题吗？非常感谢。

Answer 1

您的网络服务器或其配置似乎已损坏。查看生成 CORS Access-Control-Allow-Headers header 的原因，因为它不允许包含换行符。

HTTP header cut in half with `urllib3.exceptions.HeaderParsingError: [MissingHeaderBodySeparatorDefect()], unparsed data`

HTTP header cut in half with `urllib3.exceptions.HeaderParsingError: [MissingHeaderBodySeparatorDefect()], unparsed data`

python

nginx

python-requests

python-3.8

urllib3