gcov 在 Clang 和 GCC 上产生不同的结果

Question

我正在尝试了解如何通过使用 CMake、googletest 和 gcov 进行测试覆盖来正确构建 C++ 项目。我想构建一个适用于任何 platform/compiler.

的通用 CMakeLists.txt

This 是我的第一次尝试。但是，如果我尝试构建项目，然后运行 lcov（生成报告），我发现使用 CLang（正确结果）或 GCC（错误结果）会得到不同的结果。请注意，我在 MacOs 上，我通过 brew (brew install gcc) 安装了 gcc。

此外，我在主 CMakeLists.txt 中使用了以下标志：

if(CODE_COVERAGE)
    SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fprofile-arcs -ftest-coverage" )
endif()

注意：如果您在我的 CMakeLists.txt 文件或 lcov 用法中发现某些东西 wrong/weird，我愿意接受任何类型的反馈！

我的图书馆

#include "library.h"

#include <iostream>

void foo(){
    std::cout << "Foo!" << std::endl;
}

void bar(int n){
    if (n > 0){
        std::cout << "n is grater than 0!" << std::endl;
    }
    else if (n < 0){
        std::cout << "n is less than 0!" << std::endl;
    }
    else{
        std::cout << "n is exactly 0!" << std::endl;
    }
}

void baz(){  // LCOV_EXCL_START
    std::cout << "Baz!" << std::endl;
}
// LCOV_EXCL_STOP

我的测试


#ifndef GCOV_TUTORIAL_TEST_LIBRARY_H
#define GCOV_TUTORIAL_TEST_LIBRARY_H

#include "../src/library.h"

#include <gtest/gtest.h>


namespace gcov_tutorial::tests {
    TEST(TestFooSuite,TestFoo){
        foo();
    }
    TEST(TestBarSuite,TestBarGreaterThanZero){
        bar(100);
    }
    TEST(TestBarSuite,TestBarEqualToZero){
        //bar(0);
    }
    TEST(TestBarSuite,TestBarLessThanZero){
        bar(-100);
    }
}

#endif //GCOV_TUTORIAL_TEST_LIBRARY_H

CLang 编译

#!/bin/bash

# Rationale: https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
set -euxo pipefail

# BASE_DIR is the project's directory, containing the src/ and tests/ folders.
BASE_DIR=$PWD
COVERAGE_FILE=coverage.info

GCOV_PATH=/usr/bin/gcov
CLANG_PATH=/usr/bin/clang
CLANGPP_PATH=/usr/bin/clang++

rm -rf build
mkdir build && cd build

# Configure
cmake -DCMAKE_C_COMPILER=$CLANG_PATH -DCMAKE_CXX_COMPILER=$CLANGPP_PATH -DCODE_COVERAGE=ON -DCMAKE_BUILD_TYPE=Release ..

# Build (for Make on Unix equivalent to `make -j $(nproc)`)
cmake --build . --config Release

# Clean-up for any previous run.
rm -f $COVERAGE_FILE
lcov --zerocounters --directory .
# Run tests
./tests/RunTests
# Create coverage report by taking into account only the files contained in src/
lcov --capture --directory tests/ -o $COVERAGE_FILE --include "$BASE_DIR/src/*" --gcov-tool $GCOV_PATH
# Create HTML report in the out/ directory
genhtml $COVERAGE_FILE --output-directory out
# Show coverage report to the terminal
lcov --list $COVERAGE_FILE
# Open HTML
open out/index.html

GCC 编译

#!/bin/bash

# Rationale: https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
set -euxo pipefail

# BASE_DIR is the project's directory, containing the src/ and tests/ folders.
BASE_DIR=$PWD
COVERAGE_FILE=coverage.info

GCOV_PATH=/usr/local/bin/gcov-11
GCC_PATH=/usr/local/bin/gcc-11
GPP_PATH=/usr/local/bin/g++-11

rm -rf build
mkdir build && cd build

# Configure
cmake -DCMAKE_C_COMPILER=$GCC_PATH -DCMAKE_CXX_COMPILER=$GPP_PATH -DCODE_COVERAGE=ON -DCMAKE_BUILD_TYPE=Release ..

# Build (for Make on Unix equivalent to `make -j $(nproc)`)
cmake --build . --config Release

# Clean-up for any previous run.
rm -f $COVERAGE_FILE
lcov --zerocounters --directory .
# Run tests
./tests/RunTests
# Create coverage report by taking into account only the files contained in src/
lcov --capture --directory tests/ -o $COVERAGE_FILE --include "$BASE_DIR/src/*" --gcov-tool $GCOV_PATH
# Create HTML report in the out/ directory
genhtml $COVERAGE_FILE --output-directory out
# Show coverage report to the terminal
lcov --list $COVERAGE_FILE
# Open HTML
open out/index.html

Answer 1

你在这里实际上问了两个问题。

为什么这两个编译器的覆盖率结果不同？
如何为代码覆盖构建 CMake 项目？

答案 1：覆盖范围差异

这里的简单回答是您在 Release 模式下构建，而不是 RelWithDebInfo 模式。默认情况下，GCC 不像 Clang 那样放入那么多调试信息。在我的系统上，将 -DCMAKE_CXX_FLAGS="-g" 添加到 build-and-run-cov-gcc.sh 脚本会产生与 Clang 相同的结果，就像在 RelWithDebInfo.

中构建一样

无论出于何种原因，Clang 似乎在默认情况下或在启用覆盖时跟踪更多调试信息。 GCC 没有这些相同的护栏。要吸取的教训是：收集覆盖率信息是一种调试形式；如果您想要准确的结果，您必须为您的编译器使用 debugging-aware 配置。

答案二：构建系统结构

在您的构建中设置 CMAKE_CXX_FLAGS 通常是一个糟糕的主意。该变量旨在成为您的构建用户注入他们自己的标志的挂钩。正如我在 , the modern approach to storing such settings is in the presets

中详述的那样

我会删除你的 top-level CMakeLists.txt 的 if (CODE_COVERAGE) 部分，然后创建以下 CMakePresets.json 文件：

{
  "version": 4,
  "cmakeMinimumRequired": {
    "major": 3,
    "minor": 23,
    "patch": 0
  },
  "configurePresets": [
    {
      "name": "gcc-coverage",
      "displayName": "Code coverage (GCC)",
      "description": "Enable code coverage on GCC-compatible compilers",
      "binaryDir": "${sourceDir}/build",
      "cacheVariables": {
        "CMAKE_BUILD_TYPE": "RelWithDebInfo",
        "CMAKE_CXX_FLAGS": "-fprofile-arcs -ftest-coverage"
      }
    }
  ],
  "buildPresets": [
    {
      "name": "gcc-coverage",
      "configurePreset": "gcc-coverage",
      "configuration": "RelWithDebInfo"
    }
  ]
}

那么你的构建脚本可以大大简化。

#!/bin/bash

# Rationale: https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
set -euxo pipefail

# Set up defaults for CC, CXX, GCOV_PATH
export CC="${CC:-gcc-11}"
export CXX="${CXX:-g++-11}"
: "${GCOV_PATH:=gcov-11}"

# Record the base directory
BASE_DIR=$PWD

# Clean up old build
rm -rf build

# Configure
cmake --preset gcc-coverage

# Build
cmake --build --preset gcc-coverage

# Enter build directory
cd build

# Clean-up counters for any previous run.
lcov --zerocounters --directory .

# Run tests
./tests/RunTests

# Create coverage report by taking into account only the files contained in src/
lcov --capture --directory tests/ -o coverage.info --include "$BASE_DIR/src/*" --gcov-tool $GCOV_PATH

# Create HTML report in the out/ directory
genhtml coverage.info --output-directory out

# Show coverage report to the terminal
lcov --list coverage.info

# Open HTML
open out/index.html

这里的关键是下面几行：

# Configure
cmake --preset gcc-coverage
# Build
cmake --build --preset gcc-coverage

此脚本现在允许您通过环境变量改变编译器和覆盖工具，并且 CMakeLists.txt 不必对正在使用的编译器做出任何假设。

在我的 (Linux) 系统上，我可以运行成功执行以下命令：

$ CC=gcc-12 CXX=g++-12 GCOV=gcov-12 ./build-and-run-cov.sh

$ CC=clang-13 CXX=clang++-13 GCOV=$PWD/llvm-cov-13.sh ./build-and-run-cov.sh

其中 llvm-cov-13.sh 是 llvm-cov-13 的包装器，用于与 --gcov-tool 标志兼容。有关详细信息，请参阅 this answer。

#!/bin/bash
exec llvm-cov-13 gcov "$@"

如您所见，现在使用了正确的标志，结果无法区分。

gcov 在 Clang 和 GCC 上产生不同的结果

gcov produces different results on Clang and GCC

c++

code-coverage

cmake

gcov

lcov