没有连接可用于为该操作提供服务：使用 Azure Redis 缓存时

Question

我有以下代码用于从缓存中获取信息。我不知道是不是我的应用程序打开了太多连接，或者这个错误是由于 azure redis 缓存的暂时性故障造成的。

这是堆栈跟踪

[RedisConnectionException: No connection is available to service this operation: GET UserProfileInformation|globaladmin@xx.onmicrosoft.com] StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl(Message message, ResultProcessor1 processor, ServerEndPoint server) in c:\TeamCity\buildAgent\workae0647004edff78\StackExchange.Redis\StackExchange\Redis\ConnectionMultiplexer.cs:1922 StackExchange.Redis.RedisBase.ExecuteSync(Message message, ResultProcessor1 processor, ServerEndPoint server) in c:\TeamCity\buildAgent\workae0647004edff78\StackExchange.Redis\StackExchange\Redis\RedisBase.cs:80 StackExchange.Redis.RedisDatabase.StringGet(RedisKey key, CommandFlags flags) in c:\TeamCity\buildAgent\workae0647004edff78\StackExchange.Redis\StackExchange\Redis\RedisDatabase.cs:1431 xx.Utils.SampleStackExchangeRedisExtensions.Get(IDatabase cache, String key) in C:\Proyectos\xx\xx\Utils\SampleStackExchangeRedisExtensions.cs:20
xx.Cache.UserProfile.GetUserProfile(String identityname) in C:\Proyectos\xx\xx\Cache\UserProfile.cs:22
x.Controllers.UserProfileController.GetPropertiesForUser() in C:\Proyectos\xx\xx\Controllers\UserProfileController.cs:16
lambda_method(Closure , ControllerBase , Object[] ) +61
System.Web.Mvc.ActionMethodDispatcher.Execute(ControllerBase controller, Object[] parameters) +14

这是代码

   public static Models.UserProfile GetUserProfile(string identityname)
        {
            /// It needs to be cached for every user because every user can have different modules enabled.

            var cachekeyname = "UserProfileInformation|" + identityname;
            IDatabase cache = CacheConnectionHelper.Connection.GetDatabase();
            Models.UserProfile userProfile = new Models.UserProfile();
            object obj = cache.Get(cachekeyname);
            string userProfileString;
            if (obj != null)
            {
                //get string from cache
                userProfileString = obj.ToString();

                //conver string to our object
                userProfile = JsonConvert.DeserializeObject<Models.UserProfile>(userProfileString);
                return userProfile;
            }
            else
            {
                #region Get User Profile from AD
                Uri serviceRoot = new Uri(SettingsHelper.AzureAdGraphApiEndPoint);
                var token = AppToken.GetAppToken();

                ActiveDirectoryClient adClient = new ActiveDirectoryClient(
                 serviceRoot,
                 async () => await AppToken.GetAppTokenAsync());

                string userObjectID = ClaimsPrincipal.Current.FindFirst("http://schemas.microsoft.com/identity/claims/objectidentifier").Value;

                Microsoft.Azure.ActiveDirectory.GraphClient.Application app = (Microsoft.Azure.ActiveDirectory.GraphClient.Application)adClient.Applications.Where(
                    a => a.AppId == SettingsHelper.ClientId).ExecuteSingleAsync().Result;
                if (app == null)
                {
                    throw new ApplicationException("Unable to get a reference to application in Azure AD.");
                }

                string requestUrl = string.Format("https://graph.windows.net/{0}/users/{1}?api-version=1.5", SettingsHelper.Tenant, identityname);
                HttpClient hc = new HttpClient();
                hc.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", token);
                HttpResponseMessage hrm = hc.GetAsync(new Uri(requestUrl)).Result;

                if (hrm.IsSuccessStatusCode)
                {
                    Models.UserProfile currentUserProfile = JsonConvert.DeserializeObject<Models.UserProfile>(hrm.Content.ReadAsStringAsync().Result);

                    //convert object to json string
                    userProfileString = JsonConvert.SerializeObject(currentUserProfile);

                    cache.Set(cachekeyname, userProfileString, TimeSpan.FromMinutes(SettingsHelper.CacheUserProfileMinutes));
                    return currentUserProfile;
                }
                else
                {
                    return null;
                }
                #endregion
            }
        }




public static class SampleStackExchangeRedisExtensions
    {
        public static T Get<T>(this IDatabase cache, string key)
        {
            return Deserialize<T>(cache.StringGet(key));
        }

        public static object Get(this IDatabase cache, string key)
        {
            return Deserialize<object>(cache.StringGet(key));
        }

        public static void Set(this IDatabase cache, string key, object value, TimeSpan expiration)
        {
            cache.StringSet(key, Serialize(value), expiration);
        }

        static byte[] Serialize(object o)
        {
            if (o == null)
            {
                return null;
            }
            BinaryFormatter binaryFormatter = new BinaryFormatter();
            using (MemoryStream memoryStream = new MemoryStream())
            {
                binaryFormatter.Serialize(memoryStream, o);
                byte[] objectDataAsStream = memoryStream.ToArray();
                return objectDataAsStream;
            }
        }

        static T Deserialize<T>(byte[] stream)
        {
            BinaryFormatter binaryFormatter = new BinaryFormatter();
            if (stream == null)
                return default(T);

            using (MemoryStream memoryStream = new MemoryStream(stream))
            {
                T result = (T)binaryFormatter.Deserialize(memoryStream);
                return result;
            }
        }

问题是： 1. 我如何控制如图所示的连接异常，以便用户不会收到错误，而是在 redis 不可用时转到数据库？ 2. 是否有重试 azure redis 缓存的瞬态故障处理？

Answer 1

我认为这些是暂时性错误。在我实现简单的重试逻辑之前，我已经在我的应用程序日志中看到了很多这样的情况。我也有很多超时。非常简单的重试逻辑，加上 syncTimeout=3000 到 redis 连接字符串解决 all 这些对我来说。

public object Get(string key)
{
    return Deserialize(Cache.StringGet(key));
}

public object GetWithRetry(string key, int wait, int retryCount)
{
    int i = 0;
    do
    {
        try
        {
            return Get(key);
        }
        catch (Exception)
        {
            if (i < retryCount + 1)
            {
                Thread.Sleep(wait);
                i++;
            }
            else throw;
        }
    }
    while (i < retryCount + 1);
    return null;
}

Answer 2

此外，Stack Exchange 客户端具有内置的重试逻辑，客户端将在其中自行重试。以下是有关配置选项的更多信息。 https://azure.microsoft.com/en-us/documentation/articles/cache-faq/#what-do-the-stackexchangeredis-configuration-options-do

Answer 3

我在我的缓存存储库中使用 Polly 重试所有操作，但出现此异常。我尝试了 Polly 的重试方法，但这是错误的决定，现在我正在使用 WaitAndRetry。使用此方法，您可以在休眠一段时间后重试操作 - 使用此方法，您可以将操作排队到 Redis

Answer 4

将你的 redis nuget 包更新到最新版本，它应该可以像我一样解决你的问题！

Answer 5

正如 Rouen 所说，这些可能是暂时的连接错误。这是使用 Polly 处理重试的完整异步示例。

        var value = await Policy 
            .Handle<RedisConnectionException>() // Possible network issue 
            .WaitAndRetryAsync(3, i => TimeSpan.FromSeconds(3)) // retry 3 times, with a 3 second delay, before giving up
            .ExecuteAsync(async () => {
                return await cache.StringGetAsync(key);
            });

Answer 6

如果您使用的是 Azure 的 redis 缓存，请检查您是否在基本层。如果是这样，只要 Microsoft 更新您的服务器，您就会收到此消息（我发现这种情况每隔几周就会发生一次）。

我的确切错误消息示例：

No connection is active/available to service this operation: GET 43da9f64-da42-4281-845b-82a7d2b7f400#Settings; It was not possible to connect to the redis server(s). Error connecting right now. To allow this multiplexer to continue retrying until it's able to connect, use abortConnect=false in your connection string or AbortOnConnectFail=false; in your code. ConnectTimeout, mc: 1/1/0, mgr: 10 of 10 available, clientName: RDA04A5E790CA0, IOCP: (Busy=1,Free=999,Min=2,Max=1000), WORKER: (Busy=0,Free=32767,Min=2,Max=32767), v: 2.2.4.27433 It was not possible to connect to the redis server(s). Error connecting right now. To allow this multiplexer to continue retrying until it's able to connect, use abortConnect=false in your connection string or AbortOnConnectFail=false; in your code. ConnectTimeout

下面的屏幕截图显示在 Azure 中运行自动诊断之后，是确凿的证据。部分推荐要点：

升级到标准层或更高层以解决问题。
实施重试策略以隐藏问题。
设置更新计划以减轻问题损害。

Answer 7

使用 Azure Redis 缓存在 Azure 上托管 - 我为修复此错误所做的唯一更改是 在提供的连接字符串 的末尾添加 ,sslprotocols=tls12来自 Azure。

我也更新到 StackExchange.Redis 2.2.88 的最新版本，但我不确定这是否与此相关。（可能不是）

在 this GitHub post 中找到了答案。

没有连接可用于为该操作提供服务：使用 Azure Redis 缓存时

No connection is available to service this operation: when using Azure Redis Cache

c#

azure

stackexchange.redis

azure-redis-cache