客户端证书未添加到请求中(证书验证)

Client certificate not getting added to the request (Certificate Verify)

我正在尝试使用客户端证书向外部生产服务器发出简单的 GET 请求。 他们已将我们的证书添加到他们的服务器,我已通过 Postman(Chrome 应用程序和 Windows 本机应用程序)和标准浏览器成功发出请求:

Postman 的 Chrome 应用程序版本使用来自 Chrome 的内置证书查找器。本机 Postman 应用程序需要一个 .crt 和一个 .key 文件,我已经 extracted from my .p12 file.

换句话说,证书在商店中成功找到,并且在从文件中使用时也有效(在 Windows 本机应用程序中,建议在 .NET 中应该是可能的)。


在 C# 中获取证书

在我简单的 C# (.NET Framework 4.5.1) 控制台应用程序中,我能够从商店(或文件)获取证书,并成功地将它用于 encrypt and decrypt a file(我接受了它意味着我可以从我的应用程序中完全访问它):

private static X509Certificate2 GetCertificate(string thumbprint)
{
    X509Store store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
    store.Open(OpenFlags.ReadOnly | OpenFlags.OpenExistingOnly);
    X509Certificate2Collection coll =
        store.Certificates.Find(X509FindType.FindByThumbprint, thumbprint,
            validOnly: true);
    X509Certificate2 certificate = coll.Count == 0 ? null : coll[0];
    return certificate;
}

申请代码

我使用 HttpClientHttpWebRequest 向服务器发出请求:

//A global setting to enable TLS1.2 which is disabled in .NET 4.5.1 and 4.5.2 by default,
//and disable SSL3 which has been deprecated for a while.
//The server I'm connecting to uses TLS1.2
ServicePointManager.SecurityProtocol &= ~SecurityProtocolType.Ssl3;
ServicePointManager.SecurityProtocol |= SecurityProtocolType.Tls11
    | SecurityProtocolType.Tls12;

X509Certificate cert = GetCertificate(thumbprint);
string url = "https://sapxi.example.com/XISOAPAdapter/MessageServlet";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.ClientCertificates.Add(cert);
request.Method = WebRequestMethods.Http.Get;

WebResponse basicResponse = request.GetResponse(); //This is where the exception is thrown
string responseString = new StreamReader(basicResponse.GetResponseStream()).ReadToEnd();

例外情况

HttpClientHttpWebRequest 都抛出相同的异常:

(WebException) The underlying connection was closed: An unexpected error occurred on a send.

(IOException) Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.

(SocketException) An existing connection was forcibly closed by the remote host


跟踪 Visual Studio

中的请求

Enabling tracing,我得到一个输出,其中找到了证书和私钥(我已经过滤掉了冗长的消息):

System.Net Error: 0 : [29136] Can't retrieve proxy settings for Uri 'https://sapxi.example.com/XISOAPAdapter/MessageServlet'. Error code: 12180.
System.Net Information: 0 : [29136] Associating HttpWebRequest#21454193 with ServicePoint#60068066
System.Net Information: 0 : [29136] Associating Connection#3741682 with HttpWebRequest#21454193
System.Net.Sockets Information: 0 : [29136] Socket#33675143 - Created connection from 192.168.168.177:56114 to 131.165.*.*:443.
System.Net Information: 0 : [29136] Connection#3741682 - Created connection from 192.168.168.177:56114 to 131.165.*.*:443.
System.Net Information: 0 : [29136] TlsStream#43332040::.ctor(host=sapxi.example.com, #certs=1)
System.Net Information: 0 : [29136] Associating HttpWebRequest#21454193 with ConnectStream#54444047
System.Net Information: 0 : [29136] HttpWebRequest#21454193 - Request: GET /XISOAPAdapter/MessageServlet HTTP/1.1

System.Net Information: 0 : [29136] ConnectStream#54444047 - Sending headers
{
Host: sapxi.example.com
Connection: Keep-Alive
}.
System.Net Information: 0 : [29136] SecureChannel#20234383::.ctor(hostname=sapxi.example.com, #clientCertificates=1, encryptionPolicy=RequireEncryption)
System.Net Information: 0 : [29136] Enumerating security packages:
System.Net Information: 0 : [29136]     Negotiate
System.Net Information: 0 : [29136]     NegoExtender
System.Net Information: 0 : [29136]     Kerberos
System.Net Information: 0 : [29136]     NTLM
System.Net Information: 0 : [29136]     TSSSP
System.Net Information: 0 : [29136]     pku2u
System.Net Information: 0 : [29136]     WDigest
System.Net Information: 0 : [29136]     Schannel
System.Net Information: 0 : [29136]     Microsoft Unified Security Protocol Provider
System.Net Information: 0 : [29136]     Default TLS SSP
System.Net Information: 0 : [29136]     CREDSSP
System.Net Information: 0 : [29136] SecureChannel#20234383 - Attempting to restart the session using the user-provided certificate:
*my certificate is here* (Issuer = CN=TRUST2408 OCES CA II, O=TRUST2408, C=DK)
System.Net Information: 0 : [29136] SecureChannel#20234383 - Left with 1 client certificates to choose from.
System.Net Information: 0 : [29136] SecureChannel#20234383 - Trying to find a matching certificate in the certificate store.
System.Net Information: 0 : [29136] SecureChannel#20234383 - Locating the private key for the certificate:
*my certificate is here*
System.Net Information: 0 : [29136] SecureChannel#20234383 - Certificate is of type X509Certificate2 and contains the private key.
System.Net Information: 0 : [29136] AcquireCredentialsHandle(package = Microsoft Unified Security Protocol Provider, intent  = Outbound, scc     = System.Net.SecureCredential)
System.Net Information: 0 : [29136] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = (null), targetName = sapxi.example.com, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
System.Net Information: 0 : [29136] InitializeSecurityContext(In-Buffer length=0, Out-Buffer length=171, returned code=ContinueNeeded).
System.Net Information: 0 : [29136] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 278cca8:6d23888, targetName = sapxi.example.com, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
System.Net Information: 0 : [29136] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=ContinueNeeded).
System.Net Information: 0 : [29136] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 278cca8:6d23888, targetName = sapxi.example.com, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
System.Net Information: 0 : [29136] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=ContinueNeeded).
System.Net Information: 0 : [29136] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 278cca8:6d23888, targetName = sapxi.example.com, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
System.Net Information: 0 : [29136] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=ContinueNeeded).
System.Net Information: 0 : [29136] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 278cca8:6d23888, targetName = sapxi.example.com, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
System.Net Information: 0 : [29136] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=CredentialsNeeded).
System.Net Information: 0 : [29136] SecureChannel#20234383 - We have user-provided certificates. The server has specified 8 issuer(s). Looking for certificates that match any of the issuers.
System.Net Information: 0 : [29136] SecureChannel#20234383 - Selected certificate:
*my certificate is here*
System.Net Information: 0 : [29136] SecureChannel#20234383 - Left with 1 client certificates to choose from.
System.Net Information: 0 : [29136] SecureChannel#20234383 - Trying to find a matching certificate in the certificate store.
System.Net Information: 0 : [29136] SecureChannel#20234383 - Locating the private key for the certificate:
*my certificate is here*
System.Net Information: 0 : [29136] SecureChannel#20234383 - Certificate is of type X509Certificate2 and contains the private key.
System.Net Information: 0 : [29136] AcquireCredentialsHandle(package = Microsoft Unified Security Protocol Provider, intent  = Outbound, scc     = System.Net.SecureCredential)
System.Net Information: 0 : [29136] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 278cca8:6d23888, targetName = sapxi.example.com, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
System.Net Information: 0 : [29136] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=349, returned code=ContinueNeeded).
System.Net.Sockets Error: 0 : [29136] Socket#33675143::UpdateStatusAfterSocketError() - ConnectionReset
System.Net.Sockets Error: 0 : [29136] Exception in Socket#33675143::Receive - An existing connection was forcibly closed by the remote host.
System.Net Error: 0 : [29136] Exception in HttpWebRequest#21454193:: - The underlying connection was closed: An unexpected error occurred on a send..

上面的部分再重复一次,最后抛出异常链。我已经用示例替换了服务器的真实 URL 和 IP。

来自

的台词

We have user-provided certificates. The server has specified 8 issuer(s). Looking for certificates that match any of the issuers.

Certificate is of type X509Certificate2 and contains the private key.

让我认为在 HttpWebRequests 的内部工作中正确找到了证书。

我无法从这个输出中看出哪里出了问题。


使用 Wireshark 调试

在 Wireshark 中,我比较了 Postman 请求和我的 C# 代码,我看到的唯一区别是 Client Verify 部分(包括整个证书)不是从 C# 发送的,而是通过邮递员(和浏览器)。

通过 Postman 和浏览器,它是这样的:

通过 C#,它看起来像这样:

对我来说,我的应用程序似乎完全忽略了客户端证书。

当我不提供客户端证书时 (//request.ClientCertificates.Add(cert)),我在 Wireshark 中得到完全相同的输出,这似乎证实了我的怀疑。在 Visual Studio 的跟踪输出中,我只得到 Left with 0 client certificates to choose from. 而没有在商店中搜索证书或类似的东西。

还值得注意的是,Wireshark 表明 Postman 成功使用了 TLS1.2,而且我的应用程序代码也在使用 TLS1.2。


创建需要证书的本地网页

我让它工作,setting up the IIS Express to require certificates然后调用它。 在页面上我可以看到 Request.ClientCertificates 属性 中的证书。 在 wireshark 中,它不发送证书验证,所以还是有些不同。

但此页面在我的本地计算机上运行,​​使用 IIS Express 提示我安装的自签名证书。我还没有使用有效证书在生产服务器上设置项目,看看它的行为是否相同。

我不确定这到底是什么意思,但我想我可以确认我没有忘记一些基本的东西,这要么是一个边缘案例,要么是 HttpWebRequest C# 中的库无法正确处理。


我还尝试了什么

.

//Automatically verifying all server certificates (don't use this in production)
ServicePointManager.ServerCertificateValidationCallback =
    (sender, certificate, chain, sslPolicyErrors) =>
{
    return true; //This is not reached
};
ServicePointManager.Expect100Continue = true;
request.AllowAutoRedirect = true;
request.PreAuthenticate = true;
request.KeepAlive = false;
request.UserAgent = null;
request.CachePolicy = new HttpRequestCachePolicy(
    HttpCacheAgeControl.MaxAge, TimeSpan.FromSeconds(0));
//And several more that I didn't expect any effect from

如果有帮助,他们的服务器是 运行 SAP XI,这是拒绝我访问的应用程序。我不知道该设置是否与其他设置有很大不同,但由于 Postman 能够成功执行请求,我不怀疑它有很大不同。在最坏的情况下,它只是一个仍然遵循标准的高于平均水平的安全协议。

我的主要想法是设置简单的 ASP page/API(需要客户端证书)并将其放在我们的生产服务器上。 另一个想法是找到 HttpClient 的替代方案。 或者更糟糕的是,创建我自己的,然后尝试复制我看到 Postman 所做的交易流程。 但基本上我 运行 没主意了。感谢任何帮助。


问题

如果我必须提出一个具体的问题,我想应该是:

如何使用 C# 中的 TLS 1.2 使用我的客户端证书向 SAP XI 服务器发出 GET 请求?


此外,我不确定是否可以透露生产服务器的 URL 或 IP。当然,你们 none 中的任何一种都可以自己连接到它,因为他们不允许您将证书添加到他们的服务器。所以恐怕这不会完全重现。我想透露服务器属于 KMD 没有什么坏处。

但是如果我可以成功连接到我自己的 page/service 并在那里看到客户端证书,那么我认为无论哪种方式我都会超过目标 post,所以我认为就是这样去。

抱歉问题太长了,但通过这种方式,我提供了大量背景研究和细节,应该有助于回答者和未来的人诊断一个非常相似的问题。我也尝试在我的问题中包含一些常见问题。

我当然会自己解决这个问题,如果没有得到任何答案。

提前致谢。

在研究如何从我的本地托管页面将套接字数据捕获到 Wireshark 时,我无意中发现了 an article saying that "Certificate Verify" isn't sent over TLS 1.2 in "newer versions of Windows"(例如 Windows 10)。

所以我将协议更改为 TLS 1.0 并且请求通过了:

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls;

使用 TLS 1.1 我得到一个例外,不像那篇文章中的人所说的那样:

(WebException) The request was aborted: Could not create SSL/TLS secure channel.

我目前没有时间研究为什么这有效,因为我已经远远落后于调试这个问题的计划,但对我来说这听起来像是一个错误,很像 another user claimed in another question

我发现 a Microsoft article 沿着这些线说:

This issue only occurs with servers that downgrade the TLS session in an ungraceful way (such as by sending a TCP reset when receiving a TLS protocol version that the server does not support)

但由于我从 TLS 1.2 开始,并且服务器明确接受 TLS 1.2(通过 Postman 和 Chrome),它一定是 TLS 1.2 协议的一小部分,未以相同方式实现或者其他的东西。我仍然不明白 Postman 本机 Windows 应用程序如何设法使用 TLS 1.2。

可能值得注意的是,Internet Explorer 首先尝试 TLS 1.2,然后在 2 次重置后(就像我的客户端),它只是降级到 TLS 1.0 并通过。对我来说,这听起来与文章中谈到的 Internet Explorer 更新非常相似:


我意识到这不是一个很好的答案(当涉及到 "why" 的细节时),但至少它给出了一个提示,说明如果遇到类似问题可以尝试什么。

如果有人理解这个问题,甚至知道我如何支持 TLS 1.2,那么我将不胜感激。

我已经解决了,伙计。 TLS1.2没有问题,只需要在HttpWebRequest class.

中设置request.UserAgent = "Take it from your broewser's request header";成员即可

如果这能帮助您解决问题,请告诉我。

我最近在使用 .Net 4.7.2 时看到了同样的问题。在我的例子中,cert.HasPrivateKey 会 return 为真,但 cert.PrivateKey 会 return 为空。修复方法是将带有私钥的证书导出为 pfx,然后将其加载回内存:

// Load the cert with private key
X509Certificate2 cert = ... 

// When the certificate is created, the private key is not associated with the object. In order
// for this to work correctly, we need to export it to a PFX and then re-import the cert.
byte[] certBytes = cert.Export(X509ContentType.Pfx);
cert = new X509Certificate2(certBytes);

在此之后,HttpClient 将成功地将证书发送到服务器。希望对你有帮助。