数以千计的 TaskCanceledException 异常

Question

在大约 24 小时内，我们在一台特定服务器上收到了数千个页面加载错误。错误采用以下形式：

Event code: 3005 
Event message: An unhandled exception has occurred. 
Event time: 4/20/2019 1:43:47 PM 
Event time (UTC): 4/20/2019 1:43:47 PM 
Event sequence: 554231 
Event occurrence: 12592 
Event detail code: 0 

Process information: 
    Process ID: 6888 
    Process name: w3wp.exe 
    Account name: IIS APPPOOL\DefaultAppPool 

Exception information: 
    Exception type: TaskCanceledException 
    Exception message: A task was canceled.
   at StackExchange.Redis.ConnectionMultiplexer.Wait(Task task) in C:\projects\stackexchange-redis\src\StackExchange.Redis\ConnectionMultiplexer.cs:line 543
   at StackExchange.Redis.RedisSubscriber.StackExchange.Redis.ISubscriber.Subscribe(RedisChannel channel, Action`2 handler, CommandFlags flags) in C:\projects\stackexchange-redis\src\StackExchange.Redis\RedisSubscriber.cs:line 471
   at C3.Code.Controls.Application.Caching.Redis.PubSub.PubSubController.SubscribeToCacheKey(RedisChannel cacheKey, Action`2 onMessageReceived) in C:\Construct.net\Source\C3Alpha2\Code\Controls\Application\Caching\Redis\PubSub\PubSubController.cs:line 45
   at C3.Code.Controls.Application.Caching.Manager.Manager.Callbacks.OnGotten[T](String cacheKey, CacheType fromType, T objectGot) in C:\Construct.net\Source\C3Alpha2\Code\Controls\Application\Caching\Manager\Callbacks.cs:line 99
   at C3.Code.Controls.Application.Caching.Manager.Manager.Get[T](String key, Func`1 getFromExternFunction, Boolean skipLocalCaches) in C:\Construct.net\Source\C3Alpha2\Code\Controls\Application\Caching\Manager\Manager.cs:line 131
   at C3.PageControls.Forums.TopicRender.Page_Load(Object sender, EventArgs e) in C:\Construct.net\Source\C3Alpha2\PageControls\Forums\TopicRender.ascx.cs:line 42
   at System.Web.UI.Control.OnLoad(EventArgs e)
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

     Thread information: 
    Thread ID: 448 
    Thread account name: IIS APPPOOL\DefaultAppPool 
    Is impersonating: False 
    Stack trace:    at StackExchange.Redis.ConnectionMultiplexer.Wait(Task task) in C:\projects\stackexchange-redis\src\StackExchange.Redis\ConnectionMultiplexer.cs:line 543
   at StackExchange.Redis.RedisSubscriber.StackExchange.Redis.ISubscriber.Subscribe(RedisChannel channel, Action`2 handler, CommandFlags flags) in C:\projects\stackexchange-redis\src\StackExchange.Redis\RedisSubscriber.cs:line 471
   at C3.Code.Controls.Application.Caching.Redis.PubSub.PubSubController.SubscribeToCacheKey(RedisChannel cacheKey, Action`2 onMessageReceived) in C:\Construct.net\Source\C3Alpha2\Code\Controls\Application\Caching\Redis\PubSub\PubSubController.cs:line 45
   at C3.Code.Controls.Application.Caching.Manager.Manager.Callbacks.OnGotten[T](String cacheKey, CacheType fromType, T objectGot) in C:\Construct.net\Source\C3Alpha2\Code\Controls\Application\Caching\Manager\Callbacks.cs:line 99
   at C3.Code.Controls.Application.Caching.Manager.Manager.Get[T](String key, Func`1 getFromExternFunction, Boolean skipLocalCaches) in C:\Construct.net\Source\C3Alpha2\Code\Controls\Application\Caching\Manager\Manager.cs:line 131
   at C3.PageControls.Forums.TopicRender.Page_Load(Object sender, EventArgs e) in C:\Construct.net\Source\C3Alpha2\PageControls\Forums\TopicRender.ascx.cs:line 42
   at System.Web.UI.Control.OnLoad(EventArgs e)
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

异常似乎在之后自行解决。我迷失了可能导致这些错误的原因。有谁知道原因吗？

我们所有的 nuget 包都是最新的，我们的网络服务器也是如此。

Answer 1

你是否在基本、标准或高级定价层 (Service Tier Comparison) 订阅或使用 Azure Cache for Redis？我相信您遇到过缓存服务中断。高级层提供 Redis 集群，这将减少任何服务中断。标准层提供用于灾难恢复的复制，但任何服务中断都需要通过应用程序中的 retry 机制来缓解。基本层仅提供单个节点，没有复制或集群功能。

此外，请确保您根据 this 教程通过类似的方法实施 HomeController class。

public ActionResult RedisCache()
{
    ViewBag.Message = "A simple example with Azure Cache for Redis on ASP.NET.";

    var lazyConnection = new Lazy<ConnectionMultiplexer>(() =>
    {
        string cacheConnection = ConfigurationManager.AppSettings["CacheConnection"].ToString();
        return ConnectionMultiplexer.Connect(cacheConnection);
    });

    // Connection refers to a property that returns a ConnectionMultiplexer
    // as shown in the previous example.
    IDatabase cache = lazyConnection.Value.GetDatabase();

    // Perform cache operations using the cache object...

    // Simple PING command
    ViewBag.command1 = "PING";
    ViewBag.command1Result = cache.Execute(ViewBag.command1).ToString();

    // Simple get and put of integral data types into the cache
    ViewBag.command2 = "GET Message";
    ViewBag.command2Result = cache.StringGet("Message").ToString();

    ViewBag.command3 = "SET Message \"Hello! The cache is working from ASP.NET!\"";
    ViewBag.command3Result = cache.StringSet("Message", "Hello! The cache is working from ASP.NET!").ToString();

    // Demonstrate "SET Message" executed as expected...
    ViewBag.command4 = "GET Message";
    ViewBag.command4Result = cache.StringGet("Message").ToString();

    // Get the client list, useful to see if connection list is growing...
    ViewBag.command5 = "CLIENT LIST";
    ViewBag.command5Result = cache.Execute("CLIENT", "LIST").ToString().Replace(" id=", "\rid=");

    lazyConnection.Value.Dispose();

    return View();
}

并且正在为缓存调用正确的 FQDN，因为如果您订阅了高级层但仅指向集群中的一个实例，这很重要。

如果您investigate this issue and find you are seeing timeout or multiple disconnect issues, the following doc对这个具体问题有一些指导。

数以千计的 TaskCanceledException 异常

Thousands of TaskCanceledException exceptions

asp.net

exception

azure

redis

stackexchange.redis