在 Java 中无法使用 WebClient 连接到特定的 https 网站
Can't connect to specific https website using WebClient in Java
总结：我创建了一个简单的代码块，过去我曾多次成功使用它，但它不适用于此网站 ("https://mid-usa.com/login") 出于某种原因。我可以从我的电脑远程登录到 443 上的 mid-usa.com，我可以在浏览器中打开这个页面。
WebClient client = new WebClient(BrowserVersion.BEST_SUPPORTED);
client.getOptions().setSSLClientProtocols(new String[] { "TLSv1.2","TLSv1.1"});
client.getOptions().setCssEnabled(false);
client.getOptions().setJavaScriptEnabled(false);
client.getOptions().setUseInsecureSSL(true);
client.getOptions().setTimeout(10000);
HtmlPage page = (HtmlPage) client.getPage("https://mid-usa.com/login");
我将 setSSLClientProtocols 设置为 TLSv1.2 的原因是，在查看网站的协议 运行 扫描器时，我认为它会有所帮助（通过阅读其他帖子），但它没有.我也只保留了 TLSv1.2，但仍然打印出以下内容：
Jul 03, 2020 3:23:28 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://mid-usa.com:443: Connection reset
Jul 03, 2020 3:23:28 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://mid-usa.com:443
Jul 03, 2020 3:23:29 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://mid-usa.com:443: Connection reset
Jul 03, 2020 3:23:29 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://mid-usa.com:443
Jul 03, 2020 3:23:29 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://mid-usa.com:443: Connection reset
Jul 03, 2020 3:23:29 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://mid-usa.com:443
Connection resetjava.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at sun.security.ssl.InputRecord.readFully(Unknown Source)
at sun.security.ssl.InputRecord.read(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)
at com.gargoylesoftware.htmlunit.httpclient.HtmlUnitSSLConnectionSocketFactory.connectSocket(HtmlUnitSSLConnectionSocketFactory.java:184)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:193)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1537)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1456)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:447)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:368)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:520)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:501)
at ScraperBase.autoLogin(ScraperBase.java:25)
at MidwayDataCollection.actionPerformed(MidwayDataCollection.java:222)
at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)
at java.awt.Component.processMouseEvent(Unknown Source)
at javax.swing.JComponent.processMouseEvent(Unknown Source)
at java.awt.Component.processEvent(Unknown Source)
at java.awt.Container.processEvent(Unknown Source)
at java.awt.Component.dispatchEventImpl(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Window.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.awt.EventQueue.access0(Unknown Source)
at java.awt.EventQueue.run(Unknown Source)
at java.awt.EventQueue.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue.run(Unknown Source)
at java.awt.EventQueue.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)
有什么想法吗？
我认为问题在于服务器想要使用的密码套件在稍后的 JDK 中被禁用。
当我通过 curl -v https://mid-usa.com >/dev/null 连接时，我得到：
* SSL connection using TLSv1.2 / DES-CBC3-SHA
这映射到 JSSE 中的 SSL_RSA_WITH_3DES_EDE_CBC_SHA (https://docs.oracle.com/cd/E19728-01/820-2550/cipher_suites.html)
我重写了您的示例以仅使用 URLConnection：
URL url = new URL("https://mid-usa.com");
URLConnection urlConnection = url.openConnection();
InputStream is = urlConnection.getInputStream();
和运行它与-Djavax.net.debug=all -Dhttps.cipherSuites="SSL_RSA_WITH_3DES_EDE_CBC_SHA"
Ignoring disabled cipher suite: SSL_RSA_WITH_3DES_EDE_CBC_SHA for TLSv1.2
No available cipher suite for TLSv1.2
main, handling exception: javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
所以我认为服务器想要使用 JDK 不允许的密码。
这个 Jira 问题看起来很相似 https://bugs.openjdk.java.net/browse/JDK-8213846 -- 服务器在发送握手后断开连接。
所以现在的问题是：如何启用该密码套件，我还没有弄清楚该怎么做！
总结：我创建了一个简单的代码块，过去我曾多次成功使用它，但它不适用于此网站 ("https://mid-usa.com/login") 出于某种原因。我可以从我的电脑远程登录到 443 上的 mid-usa.com，我可以在浏览器中打开这个页面。
WebClient client = new WebClient(BrowserVersion.BEST_SUPPORTED);
client.getOptions().setSSLClientProtocols(new String[] { "TLSv1.2","TLSv1.1"});
client.getOptions().setCssEnabled(false);
client.getOptions().setJavaScriptEnabled(false);
client.getOptions().setUseInsecureSSL(true);
client.getOptions().setTimeout(10000);
HtmlPage page = (HtmlPage) client.getPage("https://mid-usa.com/login");
我将 setSSLClientProtocols 设置为 TLSv1.2 的原因是，在查看网站的协议 运行 扫描器时，我认为它会有所帮助（通过阅读其他帖子），但它没有.我也只保留了 TLSv1.2，但仍然打印出以下内容：
Jul 03, 2020 3:23:28 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://mid-usa.com:443: Connection reset
Jul 03, 2020 3:23:28 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://mid-usa.com:443
Jul 03, 2020 3:23:29 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://mid-usa.com:443: Connection reset
Jul 03, 2020 3:23:29 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://mid-usa.com:443
Jul 03, 2020 3:23:29 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://mid-usa.com:443: Connection reset
Jul 03, 2020 3:23:29 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://mid-usa.com:443
Connection resetjava.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at sun.security.ssl.InputRecord.readFully(Unknown Source)
at sun.security.ssl.InputRecord.read(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)
at com.gargoylesoftware.htmlunit.httpclient.HtmlUnitSSLConnectionSocketFactory.connectSocket(HtmlUnitSSLConnectionSocketFactory.java:184)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:193)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1537)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1456)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:447)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:368)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:520)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:501)
at ScraperBase.autoLogin(ScraperBase.java:25)
at MidwayDataCollection.actionPerformed(MidwayDataCollection.java:222)
at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)
at java.awt.Component.processMouseEvent(Unknown Source)
at javax.swing.JComponent.processMouseEvent(Unknown Source)
at java.awt.Component.processEvent(Unknown Source)
at java.awt.Container.processEvent(Unknown Source)
at java.awt.Component.dispatchEventImpl(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Window.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.awt.EventQueue.access0(Unknown Source)
at java.awt.EventQueue.run(Unknown Source)
at java.awt.EventQueue.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue.run(Unknown Source)
at java.awt.EventQueue.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)
有什么想法吗？
我认为问题在于服务器想要使用的密码套件在稍后的 JDK 中被禁用。
当我通过 curl -v https://mid-usa.com >/dev/null 连接时，我得到：
* SSL connection using TLSv1.2 / DES-CBC3-SHA
这映射到 JSSE 中的 SSL_RSA_WITH_3DES_EDE_CBC_SHA (https://docs.oracle.com/cd/E19728-01/820-2550/cipher_suites.html)
我重写了您的示例以仅使用 URLConnection：
URL url = new URL("https://mid-usa.com");
URLConnection urlConnection = url.openConnection();
InputStream is = urlConnection.getInputStream();
和运行它与-Djavax.net.debug=all -Dhttps.cipherSuites="SSL_RSA_WITH_3DES_EDE_CBC_SHA"
Ignoring disabled cipher suite: SSL_RSA_WITH_3DES_EDE_CBC_SHA for TLSv1.2
No available cipher suite for TLSv1.2
main, handling exception: javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
所以我认为服务器想要使用 JDK 不允许的密码。
这个 Jira 问题看起来很相似 https://bugs.openjdk.java.net/browse/JDK-8213846 -- 服务器在发送握手后断开连接。
所以现在的问题是：如何启用该密码套件，我还没有弄清楚该怎么做！