从 webAPI 访问调用 GetSecretAsync 时偶发 500 错误

Question

我们有一个 webapi、Azure SQL 数据库和 keyvault 托管在 azure 上。一切都很好，直到大约一周前。我们不得不删除并重新创建资源组。在此之后我们开始获得零星的 500 "internal server error" 在进一步调试时，我们发现 KeyVaultClient GetSecretAsync 抛出以下错误：

2017-10-04 15:08:06:265 - [7664:40] - 错误 - MyService.Helper.ConnectionFactory -

出现一个或多个错误。在 System.Threading.Tasks.Task`1.GetResultCore（布尔 waitCompletionNotification）在 Common.KeyVaultReader.KeyVaultHelper.GetSecret（字符串 secretName）在 syngo.DataCloud.CD.Service.Helper.ConnectionFactory..ctor（IKeyVaultHelper keyVaultHelper）

但令人困惑的是，如果我们部署到另一个端点，相同的代码也能正常工作。即使在同一个部署中，它也适用于一组 GET/POST 调用，但对另一组调用失败。

我不知道如何修复它。

KeyvaultHelper GetSecret 代码如下：

using (var kvClient = new KeyVaultClient(GetToken))
{
  var secret = Task.Run(async () =>
  {
    return await kvClient.GetSecretAsync(secretURI, secretName).ConfigureAwait(false);
  });
  return secret.Result.Value;
}

更新：

Aggregate exception as below:

2017-10-04 15:18:47:233 - [7664:86] - ERROR  - MyHelper.ExceptionHandling.GlobalExceptionLogger - WebAPI exception : System.AggregateException: One or more errors occurred. ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 104.42.136.180:443
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Rest.RetryDelegatingHandler.<>c__DisplayClass11_0.<<SendAsync>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Rest.RetryDelegatingHandler.<SendAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.KeyVault.KeyVaultClient.<GetSecretWithHttpMessagesAsync>d__61.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.KeyVault.KeyVaultClientExtensions.<GetSecretAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Common.KeyVaultReader.KeyVaultHelper.<>c__DisplayClass7_0.<<GetSecret>b__0>d.MoveNext()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Common.KeyVaultReader.KeyVaultHelper.GetSecret(String secretName)
   at Common.Authentication.Entitlement.EntitleAttribute.CheckFeatureSupported(String strToken, List`1 features)
   at System.Web.Http.AuthorizeAttribute.OnAuthorization(HttpActionContext actionContext)
   at System.Web.Http.Filters.AuthorizationFilterAttribute.OnAuthorizationAsync(HttpActionContext actionContext, CancellationToken cancellationToken)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Dispatcher.HttpControllerDispatcher.<SendAsync>d__1.MoveNext()
---> (Inner Exception #0) System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 104.42.136.180:443
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Rest.RetryDelegatingHandler.<>c__DisplayClass11_0.<<SendAsync>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Rest.RetryDelegatingHandler.<SendAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.KeyVault.KeyVaultClient.<GetSecretWithHttpMessagesAsync>d__61.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.KeyVault.KeyVaultClientExtensions.<GetSecretAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Common.KeyVaultReader.KeyVaultHelper.<>c__DisplayClass7_0.<<GetSecret>b__0>d.MoveNext()<---

Answer 1

我们无法弄清楚问题出在哪里。但是由于 keyvault 访问最初有效并且仅在大约 40/50 次调用后失败，我们决定使用 system.runtime.caching.memorycache

缓存秘密

这似乎解决了问题。

但是出现了关于密钥保管库访问限制的问题。 keyvault 是否对同时访问秘密有限制？

从 webAPI 访问调用 GetSecretAsync 时偶发 500 错误

Sporadic 500 error when accessing calling GetSecretAsync from webAPI

c#

azure

asp.net-web-api

azure-keyvault