WebClient.DownloadString 使用 HTTPS "The authentication or decryption has failed"
WebClient.DownloadString with HTTPS "The authentication or decryption has failed"
我正在 Windows10 使用 Unity Engine(2017.2.0f3) 创建游戏试图从网页中获取 HTML。据我所知，Unity 使用“Mono”C# 编译器和 运行time v2.0（根据运行ningmono --version在Unity\Editor\Data\Mono\bin的结果），还有v5.0.1（根据mono --version的结果70=]宁 mono --version 在 Unity\Editor\Data\MonoBleedingEdge\bin)。我目前正在使用 HTML Agility Pack 进行解析。尝试访问 HTTP 安全网站时，例如 dictionary.com, everything works as expected, but no matter what I do, I cannot access an HTTPS secured website, such as urbandictionary.com(Edit: Apparently, the problem is specific to this site, because after importing certificates using mozroots, github.com can be accessed), and always receive the exception: TlsException: The authentication or decryption has failed. I am aware that the HTML Agility Pack does not support HTTPS, and have wrote my code according to . I've tried the solution described here and here，但无济于事。我已经尝试 运行ning mozroots，我位于 PathTo/Unity/Editor/Data/MonoBleedingEdge/lib/mono/4.5/mozroots，带有 --import、--sync 和 --quiet 标志，但是抛出相同的异常（如果这确实有效，我怀疑它的可移植性，但也许我可以实现我自己的 mozroots 版本，正如评论所建议的那样）。另外，我被告知 Unity 使用的 Mono 版本不支持 TLS 1.2 ，这是有问题的。如果没有解决这个问题，有什么解决方法吗？
注意：为清楚起见进行了删减
class MyWebClient : WebClient
{
protected override WebRequest GetWebRequest(Uri address)
{
HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
return request;
}
}
class GetHtml
{
public static void Get(string url)
{
string documentString = new MyWebClient().DownloadString(url);
...
}
}
class CallingClass
{
public void CallingMethod()
{
ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => true;
//With or without the above line ^, the same TlsException is thrown.
GetHtml.Get("https://www.urbandictionary.com/define.php?term=example");
//HTTP works properly, but HTTPS, as in this example, does not.
}
}
日志如下：
TlsException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.RecordProtocol.ProcessAlert (AlertLevel alertLevel, AlertDescription alertDesc)
Mono.Security.Protocol.Tls.RecordProtocol.InternalReceiveRecordCallback (IAsyncResult asyncResult)
Rethrow as IOException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.SslStreamBase.AsyncHandshakeCallback (IAsyncResult asyncResult)
Rethrow as WebException: Error getting response stream (Write: The authentication or decryption has failed.): SendFailure
System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult)
System.Net.HttpWebRequest.GetResponse ()
System.Net.WebClient.GetWebResponse (System.Net.WebRequest request)
System.Net.WebClient.DownloadDataCore (System.Uri address, System.Object userToken)
编辑：我尝试更新 运行time 版本。唯一的选择是在 Project Settings->Player 菜单中将 .Net 3.5 更改为 4.6 ，这是实验性的。当代码为 运行 时，我仍然得到异常，但输出日志有点不同。
TlsException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.RecordProtocol.EndReceiveRecord (System.IAsyncResult asyncResult) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Mono.Security.Protocol.Tls.SslClientStream.SafeEndReceiveRecord (System.IAsyncResult ar, System.Boolean ignoreEmpty) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Mono.Security.Protocol.Tls.SslClientStream.NegotiateAsyncWorker (System.IAsyncResult result) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Rethrow as IOException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.SslClientStream.EndNegotiateHandshake (System.IAsyncResult result) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Mono.Security.Protocol.Tls.SslStreamBase.AsyncHandshakeCallback (System.IAsyncResult asyncResult) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Rethrow as IOException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.SslStreamBase.EndRead (System.IAsyncResult asyncResult) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Mono.Net.Security.Private.LegacySslStream.EndAuthenticateAsClient (System.IAsyncResult asyncResult) (at <344dc4d3f1ad41809df78607b6121a41>:0)
Mono.Net.Security.Private.LegacySslStream.AuthenticateAsClient (System.String targetHost, System.Security.Cryptography.X509Certificates.X509CertificateCollection clientCertificates, System.Security.Authentication.SslProtocols enabledSslProtocols, System.Boolean checkCertificateRevocation) (at <344dc4d3f1ad41809df78607b6121a41>:0)
Mono.Net.Security.MonoTlsStream.CreateStream (System.Byte[] buffer) (at <344dc4d3f1ad41809df78607b6121a41>:0)
System.Net.WebConnection.CreateStream (System.Net.HttpWebRequest request) (at <344dc4d3f1ad41809df78607b6121a41>:0)
Rethrow as WebException: Error: SecureChannelFailure (The authentication or decryption has failed.)
System.Net.WebClient.DownloadDataInternal (System.Uri address, System.Net.WebRequest& request) (at <344dc4d3f1ad41809df78607b6121a41>:0)
System.Net.WebClient.DownloadString (System.Uri address) (at <344dc4d3f1ad41809df78607b6121a41>:0)
System.Net.WebClient.DownloadString (System.String address) (at <344dc4d3f1ad41809df78607b6121a41>:0)
此问题是由 HTTP 协议引起的。如果服务器使用安全 HTTP（实际上是 HTTPS），当底层协议无法处理 X509 Certificate.
时，可能会产生错误
尝试更改此行：
HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
至：
HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
request.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => true;
或者，或者，使用以下方法将验证回调应用于全局过滤器：
ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => true;
在您的代码中创建任何 HttpWebRequest 对象之前。
如果您想更仔细地处理证书验证，请按此处所示实施回调：https://msdn.microsoft.com/en-us/library/office/dd633677
这应该可以解决您的问题。
经过严格的研究，Unity(2017.2.0f3)目前使用的Mono版本不支持TLS 1.2，正如开发人员所解释的 here。此外，对异常日志的仔细检查表明，在内部，正在使用旧后端：Mono.Net.Security.Private.LegacySslStream 而不是 Mono.Net.Security.Private.SslStream。截至目前，唯一的解决方案将涉及第三方库或变通方法。
感谢@Evk 为我指明了正确的方向。
我正在 Windows10 使用 Unity Engine(2017.2.0f3) 创建游戏试图从网页中获取 HTML。据我所知，Unity 使用“Mono”C# 编译器和 运行time v2.0（根据运行ningmono --version在Unity\Editor\Data\Mono\bin的结果），还有v5.0.1（根据mono --version的结果70=]宁 mono --version 在 Unity\Editor\Data\MonoBleedingEdge\bin)。我目前正在使用 HTML Agility Pack 进行解析。尝试访问 HTTP 安全网站时，例如 dictionary.com, everything works as expected, but no matter what I do, I cannot access an HTTPS secured website, such as urbandictionary.com(Edit: Apparently, the problem is specific to this site, because after importing certificates using mozroots, github.com can be accessed), and always receive the exception: TlsException: The authentication or decryption has failed. I am aware that the HTML Agility Pack does not support HTTPS, and have wrote my code according to mozroots，我位于 PathTo/Unity/Editor/Data/MonoBleedingEdge/lib/mono/4.5/mozroots，带有 --import、--sync 和 --quiet 标志，但是抛出相同的异常（如果这确实有效，我怀疑它的可移植性，但也许我可以实现我自己的 mozroots 版本，正如评论所建议的那样）。另外，我被告知 Unity 使用的 Mono 版本不支持 TLS 1.2 ，这是有问题的。如果没有解决这个问题，有什么解决方法吗？
注意：为清楚起见进行了删减
class MyWebClient : WebClient
{
protected override WebRequest GetWebRequest(Uri address)
{
HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
return request;
}
}
class GetHtml
{
public static void Get(string url)
{
string documentString = new MyWebClient().DownloadString(url);
...
}
}
class CallingClass
{
public void CallingMethod()
{
ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => true;
//With or without the above line ^, the same TlsException is thrown.
GetHtml.Get("https://www.urbandictionary.com/define.php?term=example");
//HTTP works properly, but HTTPS, as in this example, does not.
}
}
日志如下：
TlsException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.RecordProtocol.ProcessAlert (AlertLevel alertLevel, AlertDescription alertDesc)
Mono.Security.Protocol.Tls.RecordProtocol.InternalReceiveRecordCallback (IAsyncResult asyncResult)
Rethrow as IOException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.SslStreamBase.AsyncHandshakeCallback (IAsyncResult asyncResult)
Rethrow as WebException: Error getting response stream (Write: The authentication or decryption has failed.): SendFailure
System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult)
System.Net.HttpWebRequest.GetResponse ()
System.Net.WebClient.GetWebResponse (System.Net.WebRequest request)
System.Net.WebClient.DownloadDataCore (System.Uri address, System.Object userToken)
编辑：我尝试更新 运行time 版本。唯一的选择是在 Project Settings->Player 菜单中将 .Net 3.5 更改为 4.6 ，这是实验性的。当代码为 运行 时，我仍然得到异常，但输出日志有点不同。
TlsException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.RecordProtocol.EndReceiveRecord (System.IAsyncResult asyncResult) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Mono.Security.Protocol.Tls.SslClientStream.SafeEndReceiveRecord (System.IAsyncResult ar, System.Boolean ignoreEmpty) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Mono.Security.Protocol.Tls.SslClientStream.NegotiateAsyncWorker (System.IAsyncResult result) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Rethrow as IOException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.SslClientStream.EndNegotiateHandshake (System.IAsyncResult result) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Mono.Security.Protocol.Tls.SslStreamBase.AsyncHandshakeCallback (System.IAsyncResult asyncResult) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Rethrow as IOException: The authentication or decryption has failed.
Mono.Security.Protocol.Tls.SslStreamBase.EndRead (System.IAsyncResult asyncResult) (at <eb1224ae7b184cd09343d47f8a05481b>:0)
Mono.Net.Security.Private.LegacySslStream.EndAuthenticateAsClient (System.IAsyncResult asyncResult) (at <344dc4d3f1ad41809df78607b6121a41>:0)
Mono.Net.Security.Private.LegacySslStream.AuthenticateAsClient (System.String targetHost, System.Security.Cryptography.X509Certificates.X509CertificateCollection clientCertificates, System.Security.Authentication.SslProtocols enabledSslProtocols, System.Boolean checkCertificateRevocation) (at <344dc4d3f1ad41809df78607b6121a41>:0)
Mono.Net.Security.MonoTlsStream.CreateStream (System.Byte[] buffer) (at <344dc4d3f1ad41809df78607b6121a41>:0)
System.Net.WebConnection.CreateStream (System.Net.HttpWebRequest request) (at <344dc4d3f1ad41809df78607b6121a41>:0)
Rethrow as WebException: Error: SecureChannelFailure (The authentication or decryption has failed.)
System.Net.WebClient.DownloadDataInternal (System.Uri address, System.Net.WebRequest& request) (at <344dc4d3f1ad41809df78607b6121a41>:0)
System.Net.WebClient.DownloadString (System.Uri address) (at <344dc4d3f1ad41809df78607b6121a41>:0)
System.Net.WebClient.DownloadString (System.String address) (at <344dc4d3f1ad41809df78607b6121a41>:0)
此问题是由 HTTP 协议引起的。如果服务器使用安全 HTTP（实际上是 HTTPS），当底层协议无法处理 X509 Certificate.
尝试更改此行：
HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
至：
HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
request.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => true;
或者，或者，使用以下方法将验证回调应用于全局过滤器：
ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => true;
在您的代码中创建任何 HttpWebRequest 对象之前。
如果您想更仔细地处理证书验证，请按此处所示实施回调：https://msdn.microsoft.com/en-us/library/office/dd633677
这应该可以解决您的问题。
经过严格的研究，Unity(2017.2.0f3)目前使用的Mono版本不支持TLS 1.2，正如开发人员所解释的 here。此外，对异常日志的仔细检查表明，在内部，正在使用旧后端：Mono.Net.Security.Private.LegacySslStream 而不是 Mono.Net.Security.Private.SslStream。截至目前，唯一的解决方案将涉及第三方库或变通方法。
感谢@Evk 为我指明了正确的方向。