单线程挂在 tomcat 7

Question

有没有人遇到tomcat7挂线程的问题？在工作时间内，我们在生产服务器上收到数百万个 HTTP 请求。 Tomcat 对所有请求都表现得很好，并在指定的时间限制内做出响应。每周只有一两次线程暂停超过 10 秒，这会导致响应延迟。

环境详情为

Java 7
Tomcat 7
分配的最大内存 8GB
Spring MVC 实现
CentOS 6.5.

不确定问题是与 Tomcat 或 Java 相关还是与虚拟机有关。请注意，启用了 GC 日志记录，它显示每分钟暂停不到一秒。我在 10 秒内没有看到 gc 日志中有任何暂停。

应用程序日志

2015-12-01 14:28:41,696|lvl=INFO|cls=identity.service.IdentityService|thd=http-apr-8080-exec-6|mod=URS|opn=[modifyPostionAssignment]|ssoTicket=ESR - N/A|requestID=C6232437-0116-0401-1201-142827450001|deviceId=0.481455058954918820af268b0.6793986896475551|sessionId=JSESSIONID0.5594741341655929|tid=HG-INT-01-0GRZI7BhTreRniamQm0esA|msg=[maintainUserMasterData] START
2015-12-01 14:28:28,226|lvl=INFO|cls=identity.service.controller.IdentityServiceController|thd=http-apr-8080-exec-6|mod=URS|opn=[modifyPostionAssignment]|ssoTicket=ESR - N/A|requestID=C6232437-0116-0401-1201-142827450001|deviceId=0.481455058954918820af268b0.6793986896475551|sessionId=JSESSIONID0.5594741341655929|tid=HG-INT-01-0GRZI7BhTreRniamQm0esA|msg=[maintainUserMasterData] START

GC 日志

2015-12-01T14:28:41.461+0000: 500601.110: Total time for which application threads were stopped: 0.2703630 seconds
2015-12-01T14:28:40.705+0000: 500600.354: Total time for which application threads were stopped: 0.2837290 seconds
2015-12-01T14:28:39.473+0000: 500599.122: Total time for which application threads were stopped: 0.0011780 seconds
2015-12-01T14:28:39.123+0000: 500598.771: Total time for which application threads were stopped: 0.0866600 seconds
2015-12-01T14:28:35.918+0000: 500595.566: Total time for which application threads were stopped: 0.0001030 seconds
2015-12-01T14:28:35.918+0000: 500595.566: Total time for which application threads were stopped: 0.0001520 seconds
2015-12-01T14:28:35.917+0000: 500595.566: Total time for which application threads were stopped: 0.0005330 seconds
2015-12-01T14:28:35.914+0000: 500595.562: Total time for which application threads were stopped: 0.0001030 seconds
2015-12-01T14:28:35.914+0000: 500595.562: Total time for which application threads were stopped: 0.0002980 seconds
2015-12-01T14:28:35.913+0000: 500595.562: Total time for which application threads were stopped: 0.0012430 seconds
2015-12-01T14:28:34.468+0000: 500594.117: Total time for which application threads were stopped: 0.0001340 seconds
2015-12-01T14:28:34.468+0000: 500594.117: Total time for which application threads were stopped: 0.0001210 seconds
2015-12-01T14:28:34.468+0000: 500594.116: Total time for which application threads were stopped: 0.0001500 seconds
2015-12-01T14:28:34.467+0000: 500594.116: Total time for which application threads were stopped: 0.0012510 seconds
2015-12-01T14:28:29.479+0000: 500589.127: Total time for which application threads were stopped: 0.0012900 seconds
2015-12-01T14:28:27.979+0000: 500587.628: Total time for which application threads were stopped: 0.0607270 seconds
2015-12-01T14:28:27.101+0000: 500586.750: Total time for which application threads were stopped: 0.0982840 seconds

Tomcat 访问日志：

[01/Dec/2015:14:28:41 +0000] remotehost=192.168.62.2 x-forwarded-for=192.168.61.11 method=POST uri=/ursbusservice/maintainUserMasterData/ qs= status=200 bytes=41365 respTime=4766 method = POST status = 200
[01/Dec/2015:14:28:41 +0000] remotehost=192.168.62.2 x-forwarded-for=192.168.61.26 method=POST uri=/ursbusservice/maintainUserMasterData/ qs= status=200 bytes=4041 respTime=13580 method = POST status = 200
[01/Dec/2015:14:28:40 +0000] remotehost=192.168.62.2 x-forwarded-for=192.168.61.13 method=POST uri=/ursbusservice/maintainUserMasterData/ qs= status=200 bytes=14204 respTime=1159 method = POST status = 200
[01/Dec/2015:14:28:39 +0000] remotehost=192.168.62.2 x-forwarded-for=192.168.61.12 method=POST uri=/ursbusservice/maintainUserMasterData/ qs= status=200 bytes=59554 respTime=13990 method = POST status = 200
[01/Dec/2015:14:28:35 +0000] remotehost=192.168.62.2 x-forwarded-for=192.168.61.10 method=POST uri=/ursbusservice/maintainUserMasterData/ qs= status=200 bytes=37664 respTime=186 method = POST status = 200
[01/Dec/2015:14:28:33 +0000] remotehost=192.168.62.2 x-forwarded-for=192.168.61.12 method=POST uri=/ursbusservice/maintainUserMasterData/ qs= status=200 bytes=42983 respTime=426 method = POST status = 200

代码片段，IdentityServiceController.java：

@RequestMapping(value = RequestMappings.MAINTAINUSER_MASTER_DATA, method = RequestMethod.POST, consumes = IdentityServiceConstants.APPLICATION_JSON)
@ResponseBody
public UserMasterVO maintainUserMasterData(@RequestBody UserMasterVO userMasterVO)
{
    ContextInfo contextInfo = userMasterVO.getContextInfo();
    ServiceContext.set(contextInfo);
    dLogger.info(IdentityServiceConstants.START, contextInfo, new Object[] { IdentityServiceConstants.Methods.MAINTAIN_USERMASTER_DATA});
    if (userMasterVO.getUuid() != null && userMasterVO.getOperation() != null)
    {
        identityService.maintainUserMasterData(userMasterVO);
    }
    else
    {
        throw new IdentityServiceException();
    }
    return userMasterVO;
}

IdentityService.java:

public void maintainUserMasterData(UserMasterVO userMasterVO) {
    ContextInfo contextInfo = userMasterVO.getContextInfo();
    ServiceContext.set(contextInfo);
    dLogger.info(IdentityServiceConstants.START, contextInfo, new Object[] {IdentityServiceConstants.Methods.MAINTAIN_USERMASTER_DATA});
    //do some operations
    dLogger.info(IdentityServiceConstants.END, contextInfo, new Object[] {IdentityServiceConstants.Methods.MAINTAIN_USERMASTER_DATA});
}

Answer 1

如果它像你说的那样罕见，那么它几乎可以是任何东西。

一些要看的地方：

是否发生了 GC？
有规律吗？
如果有计时模式，请查看当时发生的情况。
您的防病毒软件多久进行一次全面扫描？
如果有数据库连接，当时表现如何？
当时有没有人登录系统？
当时是否记录了任何系统错误？
您的日志文件有多大？日志文件夹中有多少个日志文件？

单线程挂在 tomcat 7

Single thread hangs on tomcat 7

java

multithreading

spring-mvc

tomcat7