Perl LWP::UserAgent、特定 https 网站的 https 代理、未知协议错误

Question

阅读我能找到的关于主题的所有内容...不知道如何让它发挥作用。

我使用最新更新的（2015 年 1 月 9 日）Active State Perl 和标准方法

my $ua=LWP::UserAgent->new();  
$ua->agent('Mozilla/5.0 (Windows NT 6.1; WOW64; rv:34.0) Gecko/20100101 Firefox/33.0');
$ua->timeout(120);
$ua->cookie_jar( {} );
$ua->proxy(['https'],'https://199.200.120.140:8089') #taken from http://proxylist.hidemyass.com/search-1308872#listable
my $response = $ua->get('https://www.comparis.ch');   #this web site I struggle with, I can connect to https://github.com for example... But I also can connect to comparis.ch through same proxy using urllib3 on Python3.4...

如果有人能解释一下如何从网站上获得 200？

的输出

响应 returns 超时（如果我使用 http 代理，或者

LWP::Protocol::https::Socket: SSL connect attempt failed error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol at E:/Perl64/lib/LWP/Protocol/http.pm line 49."

如果我尝试

LWP::UserAgent->new(ssl_opts => { verify_hostname => 0, SSL_version => 'SSLv3' });

然后我得到

LWP::Protocol::https::Socket: SSL connect attempt failed error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number at E:/Perl64/lib/LWP/Protocol/http.pm line 49.

我认为某些地区的网站连接超时，所以必须使用来自美国或西欧的代理。

根据要求，我添加了调试输出（对于简单的 UserAgent->new() -- 无选项），它尝试来自 http://proxylist.hidemyass.com/search-1305502#listable 的两个代理 192.3.121.204:3128 和 64.31.22.131:7808。第一个似乎刚刚超时（很可能被阻止）但第二个显示不同的行为。

perl -MIO::Socket::SSL=debug4 test.pl

DEBUG: .../IO/Socket/SSL.pm:2555: new ctx 61267216
DEBUG: .../IO/Socket/SSL.pm:539: socket not yet connected
DEBUG: .../IO/Socket/SSL.pm:1769: IO::Socket::INET configuration failed
DEBUG: .../IO/Socket/SSL.pm:2588: free ctx 61267216 open=61267216
DEBUG: .../IO/Socket/SSL.pm:2593: free ctx 61267216 callback
DEBUG: .../IO/Socket/SSL.pm:2600: OK free ctx 61267216
500 Can't connect to 192.3.121.204:3128 (10060)
Content-Type: text/plain
Client-Date: Fri, 09 Jan 2015 10:15:18 GMT
Client-Warning: Internal response

Can't connect to 192.3.121.204:3128 (10060)

LWP::Protocol::https::Socket: connect: 10060 at E:/Perl64/lib/LWP/Protocol/http.pm line 49.

DEBUG: .../IO/Socket/SSL.pm:2555: new ctx 61267216
DEBUG: .../IO/Socket/SSL.pm:539: socket not yet connected
DEBUG: .../IO/Socket/SSL.pm:541: socket connected
DEBUG: .../IO/Socket/SSL.pm:563: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:599: not using SNI because hostname is unknown
DEBUG: .../IO/Socket/SSL.pm:650: set socket to non-blocking to enforce timeout=120
DEBUG: .../IO/Socket/SSL.pm:663: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:673: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:683: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:703: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:663: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:1780: SSL connect attempt failed

DEBUG: .../IO/Socket/SSL.pm:1785: SSL connect attempt failed error:140770FC:SSL     routines:SSL23_GET_SERVER_HELLO:unknown protocol
DEBUG: .../IO/Socket/SSL.pm:669: fatal SSL error: SSL connect attempt failed error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
DEBUG: .../IO/Socket/SSL.pm:1769: IO::Socket::INET configuration failed
DEBUG: .../IO/Socket/SSL.pm:2588: free ctx 61267216 open=61267216
DEBUG: .../IO/Socket/SSL.pm:2593: free ctx 61267216 callback
DEBUG: .../IO/Socket/SSL.pm:2600: OK free ctx 61267216
500 Can't connect to 64.31.22.131:7808
Content-Type: text/plain
Client-Date: Fri, 09 Jan 2015 10:15:19 GMT
Client-Warning: Internal response

Can't connect to 64.31.22.131:7808

LWP::Protocol::https::Socket: SSL connect attempt failed error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol at E:/Perl64/lib/LWP/Protocol/http.pm line 49.

根据 Stefan 的建议，我使用 http:// 作为代理地址（而不是 https），这产生了如下更好的结果，而且，它在 Ubuntu 下完美运行，但在 windows 中不行。 ..

DEBUG: .../IO/Socket/SSL.pm:2555: new ctx 65664688
DEBUG: .../IO/Socket/SSL.pm:1354: start handshake
DEBUG: .../IO/Socket/SSL.pm:563: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:599: not using SNI because hostname is unknown
DEBUG: .../IO/Socket/SSL.pm:650: set socket to non-blocking to enforce timeout=120
DEBUG: .../IO/Socket/SSL.pm:663: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:673: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:683: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:703: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:663: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:673: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:683: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:703: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:663: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:673: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:683: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:703: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:663: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:673: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:683: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:703: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:2411: ok=1 cert=69181312
DEBUG: .../IO/Socket/SSL.pm:2411: ok=1 cert=69943488
DEBUG: .../IO/Socket/SSL.pm:2411: ok=1 cert=69943136
DEBUG: .../IO/Socket/SSL.pm:1559: scheme=www cert=69943136
DEBUG: .../IO/Socket/SSL.pm:1569: identity=www.comparis.ch cn=www.comparis.ch alt=2 www.comparis.ch 2 it.comparis.ch 2 en.comparis.ch 2 fr.comparis.ch
DEBUG: .../IO/Socket/SSL.pm:663: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:673: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:683: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:703: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:663: Net::SSLeay::connect -> 1
DEBUG: .../IO/Socket/SSL.pm:718: ssl handshake done
DEBUG: .../IO/Socket/SSL.pm:2588: free ctx 65664688 open=65664688
DEBUG: .../IO/Socket/SSL.pm:2593: free ctx 65664688 callback
DEBUG: .../IO/Socket/SSL.pm:2600: OK free ctx 65664688
500 Status read failed:A non-blocking socket operation could not be completed
immediately
Content-Type: text/plain
Client-Date: Fri, 09 Jan 2015 16:49:31 GMT
Client-Warning: Internal response

Status read failed: A non-blocking socket operation could not be completed immediately
at E:/Perl64/lib/Net/HTTP/Methods.pm line 276.

我想现在有点不同了，但我在网上搜索了一下，找不到任何答案。

Answer 1

我想我明白你的问题是什么了。当您执行 https 代理请求时，您不是使用 HTTP 协议连接到代理，而是使用 HTTP 协议然后发出 CONNECT 请求以构建隧道。所以代理的 URL 必须是 http:// 而不是 https://:

$ua->proxy(['https'],'http://199.200.120.140:8089');
                      ^^^^ http:// instead of https://

除此之外，请确保 LWP::UserAgent 和 LWP::Protocol::https 至少为 6.06 版，因为它们添加了对代理 HTTPS 流量的适当支持。但我认为版本应该足够新。

编辑：由于您是运行 Windows 上的代码，使用的是 IO::Socket::SSL 的最新版本，因此您触发了一个与新添加的对非阻塞 SSL 的支持相关的错误Windows 中的套接字导致 "Status read failed: A non-blocking socket operation could not be completed immediately"。有关详细信息和修复，请参阅 https://github.com/libwww-perl/net-http/pull/11。

Answer 2

感谢 Steffen，解决方案就在那里 https://github.com/libwww-perl/net-http/pull/11 我试过了，它在我的情况下工作得很好。

Perl LWP::UserAgent、特定 https 网站的 https 代理、未知协议错误

Perl LWP::UserAgent, https proxy to specific https web site, unknown protocol error

perl

proxy