单线程 Java 应用程序启动超过 70 个线程
Single-threaded Java application starts over 70 threads
我刚刚在我们的计算服务器上启动我的单线程应用程序时遇到了 (openJDK) JVM 的一个非常奇怪的行为:
JVM 启动了很多(真的很多)线程,这些线程大大增加了所有处理器上的 CPU 负载,对此我没有合适的解释 [见下图]。
我还测试了一些东西:
- 我还在具有约 200GB RAM 的 Amazon EC2 实例上尝试了 运行 它并得到了类似的结果,只是少了几个线程。
- 我没有使用
java -Xms4096M -Xmx100000M -jar ...
启动应用程序,而是尝试了更小的参数,例如 java -Xms1024M -Xmx1024M -jar ...
,但没有任何改变。
有没有其他人经历过这种行为?
或者更好:有人可以向我解释这里发生了什么吗?
编辑:
我现在已经尝试了您建议的一些事情,并且确实得到了一些有价值和无价值的输出。
jstack PID
不起作用,建议我使用 jstack -F PID
,其主要过程的输出如下。
kill -3 PID
不输出任何东西。
Attaching to process ID 39041, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.40-b25
Deadlock Detection:
No deadlocks found.
Thread 39090: (state = BLOCKED)
Thread 39089: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Interpreted frame)
- java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=143 (Interpreted frame)
- java.lang.ref.ReferenceQueue.remove() @bci=2, line=164 (Interpreted frame)
- java.lang.ref.Finalizer$FinalizerThread.run() @bci=36, line=209 (Interpreted frame)
Thread 39088: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Interpreted frame)
- java.lang.Object.wait() @bci=2, line=502 (Interpreted frame)
- java.lang.ref.Reference$ReferenceHandler.run() @bci=36, line=157 (Interpreted frame)
Thread 39043: (state = BLOCKED)
- java.lang.AbstractStringBuilder.<init>(int) @bci=6, line=68 (Compiled frame)
- java.lang.StringBuilder.<init>(java.lang.String) @bci=8, line=112 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=526, line=99 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=413, line=86 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=80, line=39 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=136, line=43 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=676, line=112 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.PMC.LoopConvert(gov.nasa.ltl.trans.Formula, int, de.unisb.react.stud.algorithm.Tree) @bci=69, line=274 (Compiled frame)
- de.unisb.react.stud.algorithm.PMC.main(java.lang.String[]) @bci=207, line=97 (Interpreted frame)
我不确定它是属于 kill 还是 jstack,但是我在 tmux 中找到了这个:
2015-04-06 10:11:49
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.40-b25 mixed mode):
"Service Thread" #23 daemon prio=9 os_prio=0 tid=0x00007f0e54040000 nid=0x98c5 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread17" #22 daemon prio=9 os_prio=0 tid=0x00007f0e54039800 nid=0x98c4 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread16" #21 daemon prio=9 os_prio=0 tid=0x00007f0e54037800 nid=0x98c3 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread15" #20 daemon prio=9 os_prio=0 tid=0x00007f0e54035000 nid=0x98c2 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread14" #19 daemon prio=9 os_prio=0 tid=0x00007f0e54033000 nid=0x98c1 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread13" #18 daemon prio=9 os_prio=0 tid=0x00007f0e54031000 nid=0x98c0 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread12" #17 daemon prio=9 os_prio=0 tid=0x00007f0e5402f000 nid=0x98bf waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread11" #16 daemon prio=9 os_prio=0 tid=0x00007f0e5402d000 nid=0x98be waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread10" #15 daemon prio=9 os_prio=0 tid=0x00007f0e5402b000 nid=0x98bd waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread9" #14 daemon prio=9 os_prio=0 tid=0x0000000001bd9000 nid=0x98bc runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread8" #13 daemon prio=9 os_prio=0 tid=0x0000000001bd7000 nid=0x98bb waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread7" #12 daemon prio=9 os_prio=0 tid=0x0000000001bd5000 nid=0x98ba waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread6" #11 daemon prio=9 os_prio=0 tid=0x0000000001bd3000 nid=0x98b9 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread5" #10 daemon prio=9 os_prio=0 tid=0x0000000001bd1000 nid=0x98b8 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread4" #9 daemon prio=9 os_prio=0 tid=0x0000000001bc6800 nid=0x98b7 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread3" #8 daemon prio=9 os_prio=0 tid=0x0000000001bc4800 nid=0x98b6 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread2" #7 daemon prio=9 os_prio=0 tid=0x0000000001bc0000 nid=0x98b5 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x0000000001bbe000 nid=0x98b4 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x0000000001bbb800 nid=0x98b3 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x0000000001bb9000 nid=0x98b2 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x0000000001b89000 nid=0x98b1 in Object.wait() [0x00007f0e5b5ba000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00007f0f73829c78> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x00007f0f73829c78> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x0000000001b87000 nid=0x98b0 in Object.wait() [0x00007f0e5b6bb000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00007f0f73829cc0> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
- locked <0x00007f0f73829cc0> (a java.lang.ref.Reference$Lock)
"main" #1 prio=5 os_prio=0 tid=0x000000000151d800 nid=0x9883 runnable [0x00007f27cbc8c000]
java.lang.Thread.State: RUNNABLE
at de.unisb.react.stud.algorithm.BMCLoopConverter.convertLoop(BMCLoopConverter.java:156)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:118)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.PMC.LoopConvert(PMC.java:274)
at de.unisb.react.stud.algorithm.PMC.main(PMC.java:97)
"VM Thread" os_prio=0 tid=0x0000000001b82000 nid=0x98af runnable
"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x0000000001533000 nid=0x9884 runnable
"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x0000000001535000 nid=0x9885 runnable
"GC task thread#2 (ParallelGC)" os_prio=0 tid=0x0000000001536800 nid=0x9886 runnable
"GC task thread#3 (ParallelGC)" os_prio=0 tid=0x0000000001538800 nid=0x9887 runnable
"GC task thread#4 (ParallelGC)" os_prio=0 tid=0x000000000153a000 nid=0x9888 runnable
"GC task thread#5 (ParallelGC)" os_prio=0 tid=0x000000000153c000 nid=0x9889 runnable
"GC task thread#6 (ParallelGC)" os_prio=0 tid=0x000000000153d800 nid=0x988a runnable
"GC task thread#7 (ParallelGC)" os_prio=0 tid=0x000000000153f800 nid=0x988b runnable
"GC task thread#8 (ParallelGC)" os_prio=0 tid=0x0000000001541000 nid=0x988c runnable
"GC task thread#9 (ParallelGC)" os_prio=0 tid=0x0000000001543000 nid=0x988d runnable
"GC task thread#10 (ParallelGC)" os_prio=0 tid=0x0000000001544800 nid=0x988e runnable
"GC task thread#11 (ParallelGC)" os_prio=0 tid=0x0000000001546800 nid=0x988f runnable
"GC task thread#12 (ParallelGC)" os_prio=0 tid=0x0000000001548000 nid=0x9890 runnable
"GC task thread#12 (ParallelGC)" os_prio=0 tid=0x0000000001548000 nid=0x9890 runnable [31/1020]
"GC task thread#13 (ParallelGC)" os_prio=0 tid=0x000000000154a000 nid=0x9891 runnable
"GC task thread#14 (ParallelGC)" os_prio=0 tid=0x000000000154b800 nid=0x9892 runnable
"GC task thread#15 (ParallelGC)" os_prio=0 tid=0x000000000154d800 nid=0x9893 runnable
"GC task thread#16 (ParallelGC)" os_prio=0 tid=0x000000000154f000 nid=0x9894 runnable
"GC task thread#17 (ParallelGC)" os_prio=0 tid=0x0000000001551000 nid=0x9895 runnable
"GC task thread#18 (ParallelGC)" os_prio=0 tid=0x0000000001552800 nid=0x9896 runnable
"GC task thread#19 (ParallelGC)" os_prio=0 tid=0x0000000001554800 nid=0x9897 runnable
"GC task thread#20 (ParallelGC)" os_prio=0 tid=0x0000000001556000 nid=0x9898 runnable
"GC task thread#21 (ParallelGC)" os_prio=0 tid=0x0000000001558000 nid=0x9899 runnable
"GC task thread#22 (ParallelGC)" os_prio=0 tid=0x0000000001559800 nid=0x989a runnable
"GC task thread#23 (ParallelGC)" os_prio=0 tid=0x000000000155b800 nid=0x989b runnable
"GC task thread#24 (ParallelGC)" os_prio=0 tid=0x000000000155d000 nid=0x989c runnable
"GC task thread#25 (ParallelGC)" os_prio=0 tid=0x000000000155f000 nid=0x989d runnable
"GC task thread#26 (ParallelGC)" os_prio=0 tid=0x0000000001560800 nid=0x989e runnable
"GC task thread#27 (ParallelGC)" os_prio=0 tid=0x0000000001562800 nid=0x989f runnable
"GC task thread#28 (ParallelGC)" os_prio=0 tid=0x0000000001564000 nid=0x98a0 runnable
"GC task thread#29 (ParallelGC)" os_prio=0 tid=0x0000000001566000 nid=0x98a1 runnable
"GC task thread#30 (ParallelGC)" os_prio=0 tid=0x0000000001567800 nid=0x98a2 runnable
"GC task thread#31 (ParallelGC)" os_prio=0 tid=0x0000000001569800 nid=0x98a3 runnable
"GC task thread#32 (ParallelGC)" os_prio=0 tid=0x000000000156b000 nid=0x98a4 runnable
"GC task thread#33 (ParallelGC)" os_prio=0 tid=0x000000000156d000 nid=0x98a5 runnable
"GC task thread#34 (ParallelGC)" os_prio=0 tid=0x000000000156e800 nid=0x98a6 runnable
"GC task thread#35 (ParallelGC)" os_prio=0 tid=0x0000000001570800 nid=0x98a7 runnable
"GC task thread#36 (ParallelGC)" os_prio=0 tid=0x0000000001572000 nid=0x98a8 runnable
"GC task thread#37 (ParallelGC)" os_prio=0 tid=0x0000000001574000 nid=0x98a9 runnable
"GC task thread#38 (ParallelGC)" os_prio=0 tid=0x0000000001575800 nid=0x98aa runnable
"GC task thread#39 (ParallelGC)" os_prio=0 tid=0x0000000001577800 nid=0x98ab runnable
"GC task thread#40 (ParallelGC)" os_prio=0 tid=0x0000000001579000 nid=0x98ac runnable
"GC task thread#41 (ParallelGC)" os_prio=0 tid=0x000000000157b000 nid=0x98ad runnable
"GC task thread#42 (ParallelGC)" os_prio=0 tid=0x000000000157c800 nid=0x98ae runnable
"VM Periodic Task Thread" os_prio=0 tid=0x00007f0e54042800 nid=0x98c6 waiting on condition
JNI global references: 13
Heap [163/1020]
PSYoungGen total 22755328K, used 10392421K [0x00007f1f96300000, 0x00007f27b9800000, 0x00007f27b9800000)
eden space 11377664K, 91% used [0x00007f1f96300000,0x00007f22107d97a8,0x00007f224ca00000)
from space 11377664K, 0% used [0x00007f224ca00000,0x00007f224ca00000,0x00007f2503100000)
to space 11377664K, 0% used [0x00007f2503100000,0x00007f2503100000,0x00007f27b9800000)
ParOldGen total 68267008K, used 68266986K [0x00007f0f4f800000, 0x00007f1f96300000, 0x00007f1f96300000)
object space 68267008K, 99% used [0x00007f0f4f800000,0x00007f1f962fa968,0x00007f1f96300000)
Metaspace used 3156K, capacity 4240K, committed 4352K, reserved 8192K
我检查了产生高 CPU 负载的 运行 进程的 PID -> 它们属于 GC 线程。
我仍然不确定单线程程序如何产生如此多的垃圾,以至于让所有这些 GC 线程都处于忙碌状态……但至少这是一个起点。
完全解释和调试代码(这有点复杂,因为它是一个研究项目)显然超出了范围。
也许最后一个问题与主题直接相关:是否可以调整 JVM 产生的 GC 线程数?
我现在将开始调试垃圾回收,例如通过使用 -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails
,所以如果您知道有用的工具或有任何提示,请告诉我。 ;)
我唯一能给你的答案是:
尽管其他答案可能暗示了什么,但您确实有确凿证据表明存在问题。当您查看 OS 级线程时,Java 线程和 OS 线程之间存在一对一的对应关系……至少在 Linux 上是这样的当前一代的 JVM。
这不是 JVM 的错。 Java 不会自发地创建大量线程;例如~70 如您所见。
(根据 Java 版本、JVM 参数等,您可能会看到十个左右的内部线程。但它们大部分时间都是空闲的...而不是 运行 所有您的监控似乎建议的时间。)
摆弄 Java 堆和堆栈大小参数不会影响启动的线程数......以一种可能解决您的问题的方式。
运行 在不同的平台 (EC2) 上 不太可能 产生任何影响。
您需要查看您的应用程序正在做什么,以及它正在使用的第三方库。除非您提供更多详细信息,否则我们无法为您提供帮助。
但是,获取线程转储(使用 jstack
或 kill -3 PID
)应该可以提供一些关于这些线程是什么以及它们是如何创建的线索。
话虽如此,您似乎确实有一个可疑的大堆,并且(可能)有大量内核。显然,吞吐量收集器的默认策略是为每个物理核心创建一个 GC 线程。然而,很难想象一个单线程应用程序可以创建足够的垃圾来让大量 GC 线程长时间忙碌的用例……除非它已经用可访问的对象填充了堆。
我刚刚在我们的计算服务器上启动我的单线程应用程序时遇到了 (openJDK) JVM 的一个非常奇怪的行为:
JVM 启动了很多(真的很多)线程,这些线程大大增加了所有处理器上的 CPU 负载,对此我没有合适的解释 [见下图]。
我还测试了一些东西:
- 我还在具有约 200GB RAM 的 Amazon EC2 实例上尝试了 运行 它并得到了类似的结果,只是少了几个线程。
- 我没有使用
java -Xms4096M -Xmx100000M -jar ...
启动应用程序,而是尝试了更小的参数,例如java -Xms1024M -Xmx1024M -jar ...
,但没有任何改变。
有没有其他人经历过这种行为?
或者更好:有人可以向我解释这里发生了什么吗?
编辑:
我现在已经尝试了您建议的一些事情,并且确实得到了一些有价值和无价值的输出。
jstack PID
不起作用,建议我使用 jstack -F PID
,其主要过程的输出如下。
kill -3 PID
不输出任何东西。
Attaching to process ID 39041, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.40-b25
Deadlock Detection:
No deadlocks found.
Thread 39090: (state = BLOCKED)
Thread 39089: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Interpreted frame)
- java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=143 (Interpreted frame)
- java.lang.ref.ReferenceQueue.remove() @bci=2, line=164 (Interpreted frame)
- java.lang.ref.Finalizer$FinalizerThread.run() @bci=36, line=209 (Interpreted frame)
Thread 39088: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Interpreted frame)
- java.lang.Object.wait() @bci=2, line=502 (Interpreted frame)
- java.lang.ref.Reference$ReferenceHandler.run() @bci=36, line=157 (Interpreted frame)
Thread 39043: (state = BLOCKED)
- java.lang.AbstractStringBuilder.<init>(int) @bci=6, line=68 (Compiled frame)
- java.lang.StringBuilder.<init>(java.lang.String) @bci=8, line=112 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=526, line=99 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=413, line=86 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=80, line=39 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=136, line=43 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=676, line=112 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.BMCLoopConverter.convert(gov.nasa.ltl.trans.Formula, int, int, int[]) @bci=654, line=111 (Compiled frame)
- de.unisb.react.stud.algorithm.PMC.LoopConvert(gov.nasa.ltl.trans.Formula, int, de.unisb.react.stud.algorithm.Tree) @bci=69, line=274 (Compiled frame)
- de.unisb.react.stud.algorithm.PMC.main(java.lang.String[]) @bci=207, line=97 (Interpreted frame)
我不确定它是属于 kill 还是 jstack,但是我在 tmux 中找到了这个:
2015-04-06 10:11:49
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.40-b25 mixed mode):
"Service Thread" #23 daemon prio=9 os_prio=0 tid=0x00007f0e54040000 nid=0x98c5 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread17" #22 daemon prio=9 os_prio=0 tid=0x00007f0e54039800 nid=0x98c4 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread16" #21 daemon prio=9 os_prio=0 tid=0x00007f0e54037800 nid=0x98c3 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread15" #20 daemon prio=9 os_prio=0 tid=0x00007f0e54035000 nid=0x98c2 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread14" #19 daemon prio=9 os_prio=0 tid=0x00007f0e54033000 nid=0x98c1 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread13" #18 daemon prio=9 os_prio=0 tid=0x00007f0e54031000 nid=0x98c0 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread12" #17 daemon prio=9 os_prio=0 tid=0x00007f0e5402f000 nid=0x98bf waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread11" #16 daemon prio=9 os_prio=0 tid=0x00007f0e5402d000 nid=0x98be waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread10" #15 daemon prio=9 os_prio=0 tid=0x00007f0e5402b000 nid=0x98bd waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread9" #14 daemon prio=9 os_prio=0 tid=0x0000000001bd9000 nid=0x98bc runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread8" #13 daemon prio=9 os_prio=0 tid=0x0000000001bd7000 nid=0x98bb waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread7" #12 daemon prio=9 os_prio=0 tid=0x0000000001bd5000 nid=0x98ba waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread6" #11 daemon prio=9 os_prio=0 tid=0x0000000001bd3000 nid=0x98b9 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread5" #10 daemon prio=9 os_prio=0 tid=0x0000000001bd1000 nid=0x98b8 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread4" #9 daemon prio=9 os_prio=0 tid=0x0000000001bc6800 nid=0x98b7 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread3" #8 daemon prio=9 os_prio=0 tid=0x0000000001bc4800 nid=0x98b6 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread2" #7 daemon prio=9 os_prio=0 tid=0x0000000001bc0000 nid=0x98b5 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x0000000001bbe000 nid=0x98b4 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x0000000001bbb800 nid=0x98b3 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x0000000001bb9000 nid=0x98b2 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x0000000001b89000 nid=0x98b1 in Object.wait() [0x00007f0e5b5ba000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00007f0f73829c78> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x00007f0f73829c78> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x0000000001b87000 nid=0x98b0 in Object.wait() [0x00007f0e5b6bb000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00007f0f73829cc0> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
- locked <0x00007f0f73829cc0> (a java.lang.ref.Reference$Lock)
"main" #1 prio=5 os_prio=0 tid=0x000000000151d800 nid=0x9883 runnable [0x00007f27cbc8c000]
java.lang.Thread.State: RUNNABLE
at de.unisb.react.stud.algorithm.BMCLoopConverter.convertLoop(BMCLoopConverter.java:156)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:118)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.BMCLoopConverter.convert(BMCLoopConverter.java:111)
at de.unisb.react.stud.algorithm.PMC.LoopConvert(PMC.java:274)
at de.unisb.react.stud.algorithm.PMC.main(PMC.java:97)
"VM Thread" os_prio=0 tid=0x0000000001b82000 nid=0x98af runnable
"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x0000000001533000 nid=0x9884 runnable
"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x0000000001535000 nid=0x9885 runnable
"GC task thread#2 (ParallelGC)" os_prio=0 tid=0x0000000001536800 nid=0x9886 runnable
"GC task thread#3 (ParallelGC)" os_prio=0 tid=0x0000000001538800 nid=0x9887 runnable
"GC task thread#4 (ParallelGC)" os_prio=0 tid=0x000000000153a000 nid=0x9888 runnable
"GC task thread#5 (ParallelGC)" os_prio=0 tid=0x000000000153c000 nid=0x9889 runnable
"GC task thread#6 (ParallelGC)" os_prio=0 tid=0x000000000153d800 nid=0x988a runnable
"GC task thread#7 (ParallelGC)" os_prio=0 tid=0x000000000153f800 nid=0x988b runnable
"GC task thread#8 (ParallelGC)" os_prio=0 tid=0x0000000001541000 nid=0x988c runnable
"GC task thread#9 (ParallelGC)" os_prio=0 tid=0x0000000001543000 nid=0x988d runnable
"GC task thread#10 (ParallelGC)" os_prio=0 tid=0x0000000001544800 nid=0x988e runnable
"GC task thread#11 (ParallelGC)" os_prio=0 tid=0x0000000001546800 nid=0x988f runnable
"GC task thread#12 (ParallelGC)" os_prio=0 tid=0x0000000001548000 nid=0x9890 runnable
"GC task thread#12 (ParallelGC)" os_prio=0 tid=0x0000000001548000 nid=0x9890 runnable [31/1020]
"GC task thread#13 (ParallelGC)" os_prio=0 tid=0x000000000154a000 nid=0x9891 runnable
"GC task thread#14 (ParallelGC)" os_prio=0 tid=0x000000000154b800 nid=0x9892 runnable
"GC task thread#15 (ParallelGC)" os_prio=0 tid=0x000000000154d800 nid=0x9893 runnable
"GC task thread#16 (ParallelGC)" os_prio=0 tid=0x000000000154f000 nid=0x9894 runnable
"GC task thread#17 (ParallelGC)" os_prio=0 tid=0x0000000001551000 nid=0x9895 runnable
"GC task thread#18 (ParallelGC)" os_prio=0 tid=0x0000000001552800 nid=0x9896 runnable
"GC task thread#19 (ParallelGC)" os_prio=0 tid=0x0000000001554800 nid=0x9897 runnable
"GC task thread#20 (ParallelGC)" os_prio=0 tid=0x0000000001556000 nid=0x9898 runnable
"GC task thread#21 (ParallelGC)" os_prio=0 tid=0x0000000001558000 nid=0x9899 runnable
"GC task thread#22 (ParallelGC)" os_prio=0 tid=0x0000000001559800 nid=0x989a runnable
"GC task thread#23 (ParallelGC)" os_prio=0 tid=0x000000000155b800 nid=0x989b runnable
"GC task thread#24 (ParallelGC)" os_prio=0 tid=0x000000000155d000 nid=0x989c runnable
"GC task thread#25 (ParallelGC)" os_prio=0 tid=0x000000000155f000 nid=0x989d runnable
"GC task thread#26 (ParallelGC)" os_prio=0 tid=0x0000000001560800 nid=0x989e runnable
"GC task thread#27 (ParallelGC)" os_prio=0 tid=0x0000000001562800 nid=0x989f runnable
"GC task thread#28 (ParallelGC)" os_prio=0 tid=0x0000000001564000 nid=0x98a0 runnable
"GC task thread#29 (ParallelGC)" os_prio=0 tid=0x0000000001566000 nid=0x98a1 runnable
"GC task thread#30 (ParallelGC)" os_prio=0 tid=0x0000000001567800 nid=0x98a2 runnable
"GC task thread#31 (ParallelGC)" os_prio=0 tid=0x0000000001569800 nid=0x98a3 runnable
"GC task thread#32 (ParallelGC)" os_prio=0 tid=0x000000000156b000 nid=0x98a4 runnable
"GC task thread#33 (ParallelGC)" os_prio=0 tid=0x000000000156d000 nid=0x98a5 runnable
"GC task thread#34 (ParallelGC)" os_prio=0 tid=0x000000000156e800 nid=0x98a6 runnable
"GC task thread#35 (ParallelGC)" os_prio=0 tid=0x0000000001570800 nid=0x98a7 runnable
"GC task thread#36 (ParallelGC)" os_prio=0 tid=0x0000000001572000 nid=0x98a8 runnable
"GC task thread#37 (ParallelGC)" os_prio=0 tid=0x0000000001574000 nid=0x98a9 runnable
"GC task thread#38 (ParallelGC)" os_prio=0 tid=0x0000000001575800 nid=0x98aa runnable
"GC task thread#39 (ParallelGC)" os_prio=0 tid=0x0000000001577800 nid=0x98ab runnable
"GC task thread#40 (ParallelGC)" os_prio=0 tid=0x0000000001579000 nid=0x98ac runnable
"GC task thread#41 (ParallelGC)" os_prio=0 tid=0x000000000157b000 nid=0x98ad runnable
"GC task thread#42 (ParallelGC)" os_prio=0 tid=0x000000000157c800 nid=0x98ae runnable
"VM Periodic Task Thread" os_prio=0 tid=0x00007f0e54042800 nid=0x98c6 waiting on condition
JNI global references: 13
Heap [163/1020]
PSYoungGen total 22755328K, used 10392421K [0x00007f1f96300000, 0x00007f27b9800000, 0x00007f27b9800000)
eden space 11377664K, 91% used [0x00007f1f96300000,0x00007f22107d97a8,0x00007f224ca00000)
from space 11377664K, 0% used [0x00007f224ca00000,0x00007f224ca00000,0x00007f2503100000)
to space 11377664K, 0% used [0x00007f2503100000,0x00007f2503100000,0x00007f27b9800000)
ParOldGen total 68267008K, used 68266986K [0x00007f0f4f800000, 0x00007f1f96300000, 0x00007f1f96300000)
object space 68267008K, 99% used [0x00007f0f4f800000,0x00007f1f962fa968,0x00007f1f96300000)
Metaspace used 3156K, capacity 4240K, committed 4352K, reserved 8192K
我检查了产生高 CPU 负载的 运行 进程的 PID -> 它们属于 GC 线程。 我仍然不确定单线程程序如何产生如此多的垃圾,以至于让所有这些 GC 线程都处于忙碌状态……但至少这是一个起点。 完全解释和调试代码(这有点复杂,因为它是一个研究项目)显然超出了范围。
也许最后一个问题与主题直接相关:是否可以调整 JVM 产生的 GC 线程数?
我现在将开始调试垃圾回收,例如通过使用 -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails
,所以如果您知道有用的工具或有任何提示,请告诉我。 ;)
我唯一能给你的答案是:
尽管其他答案可能暗示了什么,但您确实有确凿证据表明存在问题。当您查看 OS 级线程时,Java 线程和 OS 线程之间存在一对一的对应关系……至少在 Linux 上是这样的当前一代的 JVM。
这不是 JVM 的错。 Java 不会自发地创建大量线程;例如~70 如您所见。
(根据 Java 版本、JVM 参数等,您可能会看到十个左右的内部线程。但它们大部分时间都是空闲的...而不是 运行 所有您的监控似乎建议的时间。)
摆弄 Java 堆和堆栈大小参数不会影响启动的线程数......以一种可能解决您的问题的方式。
运行 在不同的平台 (EC2) 上 不太可能 产生任何影响。
您需要查看您的应用程序正在做什么,以及它正在使用的第三方库。除非您提供更多详细信息,否则我们无法为您提供帮助。
但是,获取线程转储(使用 jstack
或 kill -3 PID
)应该可以提供一些关于这些线程是什么以及它们是如何创建的线索。
话虽如此,您似乎确实有一个可疑的大堆,并且(可能)有大量内核。显然,吞吐量收集器的默认策略是为每个物理核心创建一个 GC 线程。然而,很难想象一个单线程应用程序可以创建足够的垃圾来让大量 GC 线程长时间忙碌的用例……除非它已经用可访问的对象填充了堆。