如何解决 Heartbeat toked longer than "00:00:01" 故障？

Question

我有一个执行 HTTP POST 的 .NetCore C# 项目。该项目是在 Kubernetes 中设置的，我注意到以下日志：

Heartbeat took longer than "00:00:01" at "02/22/2020 15:43:45 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:43:46 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:43:47 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:43:48 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:43:49 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:43:50 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:43:51 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:43:52 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:43:53 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:43:54 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:43:55 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:43:56 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:44:33 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:44:34 +00:00".
warn: Microsoft.AspNetCore.Server.Kestrel[22]
      Heartbeat took longer than "00:00:01" at "02/22/2020 15:44:35 +00:00".

经过一些初步研究，这似乎是线程池饥饿的常见结果。因此，去年 11 月，我将 post 设为异步，并且还记录了最大线程数和可用线程数，如下用于监视目的：

ThreadPool.GetMaxThreads(out int workerThreads, out int completionPortThreads);
ThreadPool.GetAvailableThreads(out int workerThreadAvailable, out int completionPortThreadsAvailable);
_log.Info(new { message = $"Max threads = {workerThreads} and Available threads = {workerThreadAvailable}" });

在过去几个月中，日志记录始终显示：最大线程数 = 32767 和可用线程数 = 32766。这看起来不错，但是，我注意到同样的心跳错误，所以我想知道这是否真的是线程池饥饿问题。可能有人知道发生了什么，如果这个错误实际上是其他原因造成的？任何 investigation/resolution 的提示都将不胜感激！

Answer 1

正如@andy 在他的回复中指出的那样，这是一个资源问题。

根据 OP 的说法，此问题的解决方案是增加服务器的 CPU 容量（垂直）或增加应用程序的实例数（水平）。

如何解决 Heartbeat toked longer than "00:00:01" 故障？

How to resolve Heartbeat took longer than "00:00:01" failure?

multithreading

threadpool

kubernetes

kestrel-http-server

asp.net-core