JVM 崩溃问题帧：Canonicalizer::do_If

Question

我在启用热部署时一直面临 JVM 崩溃（在启动时使用以下 java 选项 JAVA_OPTS -Xmx4096m -XX:MetaspaceSize=512m -XX:+HeapDumpOnOutOfMemoryError -XX :HeapDumpPath=crash -XX:ThreadStackSize=512 -XX:+UseConcMarkSweepGC -XX:ParallelGCThreads=5 -XX:NewRatio=2 -XX:+UnlockDiagnosticVMOptions -XX:-UseLoopPredicate -Xdebug -Xrunjdwp:transport=dt_socket,地址=$DEBUG_PORT,server=y,suspend=n -XX:NewRatio=2 -Dspringloaded.synchronize=true JAVA_OPTS=`echo $JAVA_OPTS -Dspringloaded.synchronize =true -javaagent:springloaded-1.2.1.jar -noverify )

环境：JDK 1.8 U 66，RHEL 6.7


    #
    # Java 运行时环境检测到致命错误：
    #
    # SIGSEGV (0xb) 在 pc=0x00007faee9a1e27c，pid=27208，tid=140379827795712
    #
    # JRE 版本：Java(TM) SE Runtime Environment (8.0_66-b17) (build 1.8.0_66-b17)
    # Java 虚拟机：Java HotSpot(TM) 64 位服务器虚拟机（25.66-b17 混合模式 linux-amd64）
    # 有问题的框架：
    # V [libjvm.so+0x35027c] Canonicalizer::do_If(如果*)+0x1c
    #
    # 写入核心转储。默认位置：core.27208
    #
    # 包含更多信息的错误报告文件保存为：
    # hs_err_pid27208.log
    # [ 计时器过期，中止... ]

Answer 1

我注意到 Java 选项列表中有 -javaagent 和 -noverify。

看起来 springloaded 代理生成了无效的字节码，而字节码验证被明确关闭。毫不奇怪，这可能会导致不可预测的结果，包括 JVM 崩溃。

这不是 JVM 问题，但很可能是 springloaded 代理中的错误。尝试删除 -noverify 选项。

-XX:-TieredCompilation 也可以解决这个特殊问题，但如果字节码未能通过验证，不要指望应用程序能够正常工作。最好远离有问题的代理库。

Answer 2

4.2.1 Crash in HotSpot Compiler Thread or Compiled Code

If the fatal error log indicates that the crash occurred in a compiler thread, then it is possible (but not always the case) that you have encountered a compiler bug. Similarly, if the crash is in compiled code then it is possible that the compiler has generated incorrect code.

In the case of the HotSpot Client VM (-client option), the compiler thread appears in the error log as CompilerThread0. With the HotSpot Server VM there are multiple compiler threads and these appear in the error log file as CompilerThread0, CompilerThread1, and AdapterThread.

Below is a fragment of an error log for a compiler bug that was encountered and fixed during the development of J2SE 5.0. The log file shows that the HotSpot Server VM is used and the crash occurred in CompilerThread1. In addition, the log file shows that the Current CompileTask was the compilation of the java.lang.Thread.setPriority method.

An unexpected error has been detected by HotSpot Virtual Machine: : Java VM: Java HotSpot(TM) Server VM (1.5-internal-debug mixed mode) : --------------- T H R E A D ---------------

Current thread (0x001e9350): JavaThread "CompilerThread1" daemon [_thread_in_vm, id=20]

Stack: [0xb2500000,0xb2580000), sp=0xb257e500, free space=505k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0xc3b13c] :

Current CompileTask: opto: 11 java.lang.Thread.setPriority(I)V (53 bytes)

--------------- P R O C E S S ---------------

Java Threads: ( => current thread ) 0x00229930 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=21] =>0x001e9350 JavaThread "CompilerThread1" daemon [_thread_in_vm, id=20] :

In this case there are two potential workarounds:
The brute force approach: change the configuration so that the application is run with the -client option to specify the HotSpot
Client VM.
Assume that the bug only occurs during the compilation of the setPriority method and exclude this method from compilation.
The first approach (to use the -client option) might be trivial to configure in some environments. In others, it might be more difficult if the configuration is complex or if the command line to configure the VM is not readily accessible. In general, switching from the HotSpot Server VM to the HotSpot Client VM also reduces the peak performance of an application. Depending on the environment, this might be acceptable until the actual issue is diagnosed and fixed.

The second approach (exclude the method from compilation) requires creating the file .hotspot_compiler in the working directory of the application. Below is an example of this file:

exclude java/lang/Thread setPriority

In general the format of this file is exclude CLASS METHOD, where CLASS is the class (fully qualified with the package name) and METHOD is the name of the method. Constructor methods are specified as and static initializers are specified as .

Note - The .hotspot_compiler file is an unsupported interface. It is documented here solely for the purposes of troubleshooting and finding a temporary workaround.

Once the application is restarted, the compiler will not attempt to compile any of the methods listed as excluded in the .hotspot_compiler file. In some cases this can provide temporary relief until the root cause of the crash is diagnosed and the bug is fixed.

In order to verify that the HotSpot VM correctly located and processed the .hotspot_compiler file that is shown in the example above, look for the following log information at runtime. Note that the file name separator is a dot, not a slash.

Excluding compile: java.lang.Thread::setPriority

Source

Answer 3

同意@apangin，在程序中你正在做字节码插入（-agent）但指定了-noverify。关闭验证后，您可能会遇到此类崩溃。

您不应在字节码插入期间使用 -noverify 或 -Xverify:none。

对于那些不熟悉字节码验证的人来说，它只是 JVM 类加载过程的一部分，用于检查代码中是否存在某些危险和不允许的行为。您可以（但不应该）通过将 -Xverify:none 或 -noverify 添加到 Java 命令行来在许多 JVM 上禁用此保护。 https://blogs.oracle.com/buck/entry/never_disable_bytecode_verification_in

JVM 崩溃问题帧：Canonicalizer::do_If

JVM Crash Problematic Frame: Canonicalizer::do_If

crash

jvm

canonical-link