请求 SSL：CERTIFICATE_VERIFY_FAILED with verify="./cabundle.crt"
requests SSL: CERTIFICATE_VERIFY_FAILED with verify="./cabundle.crt"
问题
简短描述：python 请求不断抛出 SSL：CERTIFICATE_VERIFY_FAILED 使用自己的 cabundle 连接到服务器时：
In [1]: import requests
In [3]: requests.get('https://activeo.monitowl.com', verify=True)
Out[3]: <Response [200]>
In [4]: requests.get('https://activeo.monitowl.com', verify="./ca_bundle.crt")
---------------------------------------------------------------------------
SSLError Traceback (most recent call last)
<ipython-input-3-c92a3091d6ce> in <module>()
----> 1 requests.get('https://activeo.monitowl.com', verify="./ca_bundle.crt")
/home/vagrant/.virtualenvs/test/local/lib/python2.7/site-packages/requests/api.pyc in get(url, params, **kwargs)
67
68 kwargs.setdefault('allow_redirects', True)
---> 69 return request('get', url, params=params, **kwargs)
70
71
/home/vagrant/.virtualenvs/test/local/lib/python2.7/site-packages/requests/api.pyc in request(method, url, **kwargs)
48
49 session = sessions.Session()
---> 50 response = session.request(method=method, url=url, **kwargs)
51 # By explicitly closing the session, we avoid leaving sockets open which
52 # can trigger a ResourceWarning in some cases, and look like a memory leak
/home/vagrant/.virtualenvs/test/local/lib/python2.7/site-packages/requests/sessions.pyc in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
463 }
464 send_kwargs.update(settings)
--> 465 resp = self.send(prep, **send_kwargs)
466
467 return resp
/home/vagrant/.virtualenvs/test/local/lib/python2.7/site-packages/requests/sessions.pyc in send(self, request, **kwargs)
571
572 # Send the request
--> 573 r = adapter.send(request, **kwargs)
574
575 # Total elapsed time of the request (approximately)
/home/vagrant/.virtualenvs/test/local/lib/python2.7/site-packages/requests/adapters.pyc in send(self, request, stream, timeout, verify, cert, proxies)
429 except (_SSLError, _HTTPError) as e:
430 if isinstance(e, _SSLError):
--> 431 raise SSLError(e, request=request)
432 elif isinstance(e, ReadTimeoutError):
433 raise ReadTimeout(e, request=request)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
In [5]: requests.get('https://activeo.monitowl.com', verify="./ca_bundle_ff.crt")
[same error]
In [7]: import ssl; ssl.OPENSSL_VERSION
Out[7]: 'OpenSSL 1.0.1k 8 Jan 2015'
In [8]: import sys; print (sys.version)
2.7.9 (default, Mar 1 2015, 12:57:24)
[GCC 4.9.2]
In [2]: requests.__version__
Out[2]: '2.7.0'
故事
我们在 MonitOwl agent to communicate with API servers. Few days ago we ordered new certificates for *.monitowl.com and deployed it to new instance, our issuer is nazwa.pl. Pointing a webbrowser to https://activeo.monitowl.com 中使用 requests 工作完美，当设置 verify="./cabundle.crt".
requests 似乎有一些问题
技术说明
在处理 ssl 的龙卷风服务器前面有一个 "transparent" haproxy 检查 SNI 并重新分配流量，配置文件的一部分：
acl app_activeo req_ssl_sni -i activeo.monitowl.com
use_backend bk_activeo if app_activeo
我们需要提供自己的 cabundle 来控制接受的内容 - 因为安全问题和部署脚本。我们到处都在使用 debian jessie。
我用 ssl checker 测试了服务器，没有任何问题，除了接受旧的加密算法..
# from issuer https://panel.nazwa.pl/uploads/ssl/nazwaSSL_SHA-2.zip
$ cat monitowlcom.crt nazwasslsha2.pem certumca-ctncasha2.pem gscasha2.pem > ./ca_bundle.crt
# exported from firefox
$ cat monitowl.com nazwaSSL CertumTrustedNetworkCA CertumGlobalServicesCASHA2 > ca_bundle_ff.crt
$ openssl verify -untrusted ./ca.crt monitowlcom.crt
monitowlcom.crt: OK
$ c_rehash ./
$ openssl s_client -CApath ./ -connect activeo.monitowl.com:443 -servername activeo.monitowl.com
CONNECTED(00000003)
depth=4 C = PL, O = Unizeto Sp. z o.o., CN = Certum CA
verify return:1
depth=3 C = PL, O = Unizeto Technologies S.A., OU = Certum Certification Authority, CN = Certum Trusted Network CA
verify return:1
depth=2 C = PL, O = Unizeto Technologies S.A., OU = Certum Certification Authority, CN = Certum Global Services CA SHA2
verify return:1
depth=1 C = PL, O = nazwa.pl S.A., OU = http://nazwa.pl, CN = nazwaSSL
verify return:1
depth=0 C = PL, CN = *.monitowl.com, emailAddress = ***@whitehats.pl
verify return:1
[...]
* Verify return code: 0 (ok)*
$ curl -I --cacert ca_bundle.crt https://activeo.monitowl.com
HTTP/1.1 200 OK
$ curl -I --cacert ca_bundle_ff.crt https://activeo.monitowl.com
HTTP/1.1 200 OK
如您所见，openssl s_client 验证连接，curl 工作正常。
在服务器端（龙卷风）：
SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:581)
我请一些朋友使用从网络浏览器导出的包对其进行测试，并且：
Python==2.7.9 + OpenSSL==1.0.1k => FAILS
Python==2.7.10 + OpenSSL==1.0.1k => FAILS
Python==2.7.9 + OpenSSL==1.0.1l => FAILS
Python==2.7.10 + OpenSSL==1.0.1p => WORKS
Python==2.7.10 + OpenSSL==1.0.2d => WORKS
Python==2.7.9 + OpenSSL 1.0.2d => WORKS
我知道 python 比 Web 浏览器更能验证证书 ，但看起来这不是这里的问题。
问题
你知道这个包有什么问题吗？
我还能检查什么？
这是一个错误？在请求中，openssl、python 或 urllib3?
不要使用您的 ca 包文件。尝试使用 verify=True 并将你的证书文件的内容粘贴到 /usr/local/lib/python2.7/dist-packages/requests/cacert.pem 的底部，位置取决于你的 Linux 版本或 dist (CentOS 、Debian 等..）。请使用 find / -name requests 找到系统中安装请求的位置，然后找出 cacert.pem.
这是因为 Requests 默认使用这个 cacert.pem 作为它自己的 ca bundle。
如果这不起作用，那么您应该获取高级颁发者证书作为证书文件。
正如 Lukasa（请求合作者之一）指出的，问题是因为 python 中对交叉签名链的支持不佳：
The problem with the SHA256 bundle is that the 'root' cert in that case is cross-signed with the SHA-1 cert, but older OpenSSL's wont like that at all. The way older OpenSSL builds a cert chain is that it builds the longest one it can out of the certs provided by the remote server, and then looks for a trusted certificate that signs that chain. If it can't find that certificate, it'll bail out, without checking whether it can build a shorter chain using its trust roots.
This is an ongoing problem with the deprecation of SHA1 root certs: see certifi/python-certifi#26. Unfortunately, the only fixes are to temporarily use the SHA1 root in your trust store or to upgrade to a newer OpenSSL version.
https://github.com/kennethreitz/requests/issues/2783
因此，作为临时修复，我在我的 cabundle 中也包含了 SHA1 路径：
cat ca_sha1.crt ca_sha256.crt > cabundle.crt
问题
简短描述：python 请求不断抛出 SSL：CERTIFICATE_VERIFY_FAILED 使用自己的 cabundle 连接到服务器时：
In [1]: import requests
In [3]: requests.get('https://activeo.monitowl.com', verify=True)
Out[3]: <Response [200]>
In [4]: requests.get('https://activeo.monitowl.com', verify="./ca_bundle.crt")
---------------------------------------------------------------------------
SSLError Traceback (most recent call last)
<ipython-input-3-c92a3091d6ce> in <module>()
----> 1 requests.get('https://activeo.monitowl.com', verify="./ca_bundle.crt")
/home/vagrant/.virtualenvs/test/local/lib/python2.7/site-packages/requests/api.pyc in get(url, params, **kwargs)
67
68 kwargs.setdefault('allow_redirects', True)
---> 69 return request('get', url, params=params, **kwargs)
70
71
/home/vagrant/.virtualenvs/test/local/lib/python2.7/site-packages/requests/api.pyc in request(method, url, **kwargs)
48
49 session = sessions.Session()
---> 50 response = session.request(method=method, url=url, **kwargs)
51 # By explicitly closing the session, we avoid leaving sockets open which
52 # can trigger a ResourceWarning in some cases, and look like a memory leak
/home/vagrant/.virtualenvs/test/local/lib/python2.7/site-packages/requests/sessions.pyc in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
463 }
464 send_kwargs.update(settings)
--> 465 resp = self.send(prep, **send_kwargs)
466
467 return resp
/home/vagrant/.virtualenvs/test/local/lib/python2.7/site-packages/requests/sessions.pyc in send(self, request, **kwargs)
571
572 # Send the request
--> 573 r = adapter.send(request, **kwargs)
574
575 # Total elapsed time of the request (approximately)
/home/vagrant/.virtualenvs/test/local/lib/python2.7/site-packages/requests/adapters.pyc in send(self, request, stream, timeout, verify, cert, proxies)
429 except (_SSLError, _HTTPError) as e:
430 if isinstance(e, _SSLError):
--> 431 raise SSLError(e, request=request)
432 elif isinstance(e, ReadTimeoutError):
433 raise ReadTimeout(e, request=request)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
In [5]: requests.get('https://activeo.monitowl.com', verify="./ca_bundle_ff.crt")
[same error]
In [7]: import ssl; ssl.OPENSSL_VERSION
Out[7]: 'OpenSSL 1.0.1k 8 Jan 2015'
In [8]: import sys; print (sys.version)
2.7.9 (default, Mar 1 2015, 12:57:24)
[GCC 4.9.2]
In [2]: requests.__version__
Out[2]: '2.7.0'
故事
我们在 MonitOwl agent to communicate with API servers. Few days ago we ordered new certificates for *.monitowl.com and deployed it to new instance, our issuer is nazwa.pl. Pointing a webbrowser to https://activeo.monitowl.com 中使用 requests 工作完美，当设置 verify="./cabundle.crt".
技术说明
在处理 ssl 的龙卷风服务器前面有一个 "transparent" haproxy 检查 SNI 并重新分配流量，配置文件的一部分：
acl app_activeo req_ssl_sni -i activeo.monitowl.com
use_backend bk_activeo if app_activeo
我们需要提供自己的 cabundle 来控制接受的内容 - 因为安全问题和部署脚本。我们到处都在使用 debian jessie。
我用 ssl checker 测试了服务器，没有任何问题，除了接受旧的加密算法..
# from issuer https://panel.nazwa.pl/uploads/ssl/nazwaSSL_SHA-2.zip
$ cat monitowlcom.crt nazwasslsha2.pem certumca-ctncasha2.pem gscasha2.pem > ./ca_bundle.crt
# exported from firefox
$ cat monitowl.com nazwaSSL CertumTrustedNetworkCA CertumGlobalServicesCASHA2 > ca_bundle_ff.crt
$ openssl verify -untrusted ./ca.crt monitowlcom.crt
monitowlcom.crt: OK
$ c_rehash ./
$ openssl s_client -CApath ./ -connect activeo.monitowl.com:443 -servername activeo.monitowl.com
CONNECTED(00000003)
depth=4 C = PL, O = Unizeto Sp. z o.o., CN = Certum CA
verify return:1
depth=3 C = PL, O = Unizeto Technologies S.A., OU = Certum Certification Authority, CN = Certum Trusted Network CA
verify return:1
depth=2 C = PL, O = Unizeto Technologies S.A., OU = Certum Certification Authority, CN = Certum Global Services CA SHA2
verify return:1
depth=1 C = PL, O = nazwa.pl S.A., OU = http://nazwa.pl, CN = nazwaSSL
verify return:1
depth=0 C = PL, CN = *.monitowl.com, emailAddress = ***@whitehats.pl
verify return:1
[...]
* Verify return code: 0 (ok)*
$ curl -I --cacert ca_bundle.crt https://activeo.monitowl.com
HTTP/1.1 200 OK
$ curl -I --cacert ca_bundle_ff.crt https://activeo.monitowl.com
HTTP/1.1 200 OK
如您所见，openssl s_client 验证连接，curl 工作正常。
在服务器端（龙卷风）：
SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:581)
我请一些朋友使用从网络浏览器导出的包对其进行测试，并且：
Python==2.7.9 + OpenSSL==1.0.1k => FAILS
Python==2.7.10 + OpenSSL==1.0.1k => FAILS
Python==2.7.9 + OpenSSL==1.0.1l => FAILS
Python==2.7.10 + OpenSSL==1.0.1p => WORKS
Python==2.7.10 + OpenSSL==1.0.2d => WORKS
Python==2.7.9 + OpenSSL 1.0.2d => WORKS
我知道 python 比 Web 浏览器更能验证证书
问题
你知道这个包有什么问题吗？ 我还能检查什么？
这是一个错误？在请求中，openssl、python 或 urllib3?
不要使用您的 ca 包文件。尝试使用 verify=True 并将你的证书文件的内容粘贴到 /usr/local/lib/python2.7/dist-packages/requests/cacert.pem 的底部，位置取决于你的 Linux 版本或 dist (CentOS 、Debian 等..）。请使用 find / -name requests 找到系统中安装请求的位置，然后找出 cacert.pem.
这是因为 Requests 默认使用这个 cacert.pem 作为它自己的 ca bundle。
如果这不起作用，那么您应该获取高级颁发者证书作为证书文件。
正如 Lukasa（请求合作者之一）指出的，问题是因为 python 中对交叉签名链的支持不佳：
The problem with the SHA256 bundle is that the 'root' cert in that case is cross-signed with the SHA-1 cert, but older OpenSSL's wont like that at all. The way older OpenSSL builds a cert chain is that it builds the longest one it can out of the certs provided by the remote server, and then looks for a trusted certificate that signs that chain. If it can't find that certificate, it'll bail out, without checking whether it can build a shorter chain using its trust roots.
This is an ongoing problem with the deprecation of SHA1 root certs: see certifi/python-certifi#26. Unfortunately, the only fixes are to temporarily use the SHA1 root in your trust store or to upgrade to a newer OpenSSL version.
https://github.com/kennethreitz/requests/issues/2783
因此，作为临时修复，我在我的 cabundle 中也包含了 SHA1 路径：
cat ca_sha1.crt ca_sha256.crt > cabundle.crt