为什么从 bazel 执行的 pytest 会话将 classname junit 属性留空？

Question

我发现我们之前在命令行直接从 pytest 会话生成的一些 junit xml 不再具有 testcase 标记的 classname 属性在相同的测试中被填充运行通过 bazel。为什么 classname 在 bazel 下生成的 xml 中返回 void，作为一个空字符串？

这是一个可重现的示例，用于演示...

项目结构

$ tree junit_explore/
junit_explore/
├── BUILD
└── test_explore.py

构建文件

# BUILD
load("@python3_deps//:requirements.bzl", "requirement")

py_test(
    name = "test_explore",
    srcs = ["test_explore.py"],
    args = [
        "--junit-xml=out.xml",
        #"--junit-prefix=THIS", # <----- uncommented in last session
    ],
    deps = [
        requirement("pytest"),
    ],
)

测试模块

# test_explore.py
import sys
import pytest

def test_pass():
    assert True

def test_fail():
    assert False


if __name__ == "__main__":
    sys.exit(pytest.main([__file__] + sys.argv[1:]))

直接 Pytest 会话

pytest test_explore.py --junit-xml=out_pytest.xml

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
    <testsuite errors="0" failures="1" hostname="HOST_X" name="pytest" skipped="0" tests="2" time="0.029" timestamp="2022-03-17T22:16:13.130304">
        <testcase classname="test_explore" file="test_explore.py" line="3" name="test_pass" time="0.000"></testcase>
        <testcase classname="test_explore" file="test_explore.py" line="6" name="test_fail" time="0.000">
            <failure message="assert False">def test_fail():
&gt;       assert False
E       assert False

test_explore.py:8: AssertionError</failure>
        </testcase>
    </testsuite>
</testsuites>

注意类名的值，它是classname="test_explore"测试模块的名称。

Bazel 驱动的 pytest 会话

bazel test //junit_explore:test_explore

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
    <testsuite errors="0" failures="1" hostname="HOST_X" name="pytest" skipped="0" tests="2" time="0.030" timestamp="2022-03-18T05:15:14.118882">
        <testcase classname="" file="../../../../../../../../../../../../workspace_foo/repo_X/junit_explore/test_explore.py" line="3" name="test_pass" time="0.000"></testcase>
        <testcase classname="" file="../../../../../../../../../../../../workspace_foo/repo_X/junit_explore/test_explore.py" line="6" name="test_fail" time="0.000">
            <failure message="assert False">def test_fail():
&gt;       assert False
E       assert False

/home/USERX/workspace_foo/repo_X/junit_explore/test_explore.py:8: AssertionError</failure>
        </testcase>
    </testsuite>
</testsuites>

注意类名的值，它是 classname="" 空字符串。

Bazel 驱动的 pytest 会话，在 BUILD 文件中未注释 JUnit 前缀

bazel test //junit_explore:test_explore

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
    <testsuite errors="0" failures="1" hostname="HOST_X" name="pytest" skipped="0" tests="2" time="0.032" timestamp="2022-03-18T05:33:10.835593">
        <testcase classname="THIS." file="../../../../../../../../../../../../workspace_foo/repo_X/junit_explore/test_explore.py" line="3" name="test_pass" time="0.000"></testcase>
        <testcase classname="THIS." file="../../../../../../../../../../../../workspace_foo/repo_X/junit_explore/test_explore.py" line="6" name="test_fail" time="0.000">
            <failure message="assert False">def test_fail():
&gt;       assert False
E       assert False

/home/USERX/workspace_foo/repo_X/junit_explore/test_explore.py:8: AssertionError</failure>
        </testcase>
    </testsuite>
</testsuites>

注意类名的值，classname="THIS." 只是命令行中给出的点分隔前缀。

这一切都发生在

巴泽尔 4.2.2-1.4
pytest 5.4.3

Answer 1

这个空类名属性的根本原因可以在 nodes.py

中 pytest nodeid 实现的以下源代码行中找到

    nodeid = self.fspath.relto(session.config.rootdir)

此 pytest discussion 的答案通过一些调试进行说明，在此操作后打印 nodeid 的值，该操作确定相对路径返回为空。原因是因为 rootdir 的值将它放在 bazel 运行文件中，而 fspath 的值将它放在 repo 克隆中。

self.fspath = "/home/USERX/repoX/junit_explore/test_explore.py"
session.config.rootdir = "/home/USERX/.cache/bazel/_bazel_USERX/59cb4fc3b17888451f2ad5df027bf8f3/execroot/workspace_foo/bazel-out/bin/junit_explore/test_explore.runfiles/workspace_foo"

如你所见，两条路径之间没有公共路径。自 5.4.3 以来，nodes.py 源中的 pytest 发生了变化，因此此行为可能会在以后的版本中消失。

（第一个答案）

我发现我可以直接在 bazel 缓存中编辑 junitxml.py 文件以添加一些调试打印。我也为我的 site-packages 本地系统 python 版本做了同样的事情。目前尚不清楚具体原因，但罪魁祸首在于 nodeid 的生成方式，它从一开始就是空的，因此 junitxml.py 中没有任何内容可以解释这种差异，在执行此逻辑之前已经发生了一些偏差。

# junitxml.py
    def record_testreport(self, testreport):
        assert not self.testcase
        print("\n===================================================\nDEBUG START")
        print("testreport.nodeid: %s" % testreport.nodeid)
        names = mangle_test_address(testreport.nodeid)
        print("names: %s" % names)
        existing_attrs = self.attrs
        print("existing_attrs: %s" % existing_attrs)
        classnames = names[:-1]
        print("classnames: %s" % classnames)
        if self.xml.prefix:
            classnames.insert(0, self.xml.prefix)
        attrs = {
            "classname": ".".join(classnames),
            "name": bin_xml_escape(names[-1]),
            "file": testreport.location[0],
        }
        if testreport.location[1] is not None:
            attrs["line"] = testreport.location[1]
        if hasattr(testreport, "url"):
            attrs["url"] = testreport.url
        self.attrs = attrs
        print("attrs: %s" % attrs)
        self.attrs.update(existing_attrs)  # restore any user-defined attributes
        print("DEBUG END")

为

提供以下内容

Bazel 执行

===================================================
DEBUG START
testreport.nodeid: ::test_pass
names: ['', 'test_pass']
existing_attrs: {}
classnames: ['']
attrs: {'classname': '', 'name': <py._xmlgen.raw object at 0x7f8114576630>, 'file': '../../../../../../../../../../../../repoX/junit_explore/test_explore.py', 'line': 4}
DEBUG END

===================================================
DEBUG START
testreport.nodeid: ::test_pass
names: ['', 'test_pass']
existing_attrs: {'classname': '', 'name': <py._xmlgen.raw object at 0x7f8114576630>, 'file': '../../../../../../../../../../../../repoX/junit_explore/test_explore.py', 'line': 4}
classnames: ['']
attrs: {'classname': '', 'name': <py._xmlgen.raw object at 0x7f81145764e0>, 'file': '../../../../../../../../../../../../repoX/junit_explore/test_explore.py', 'line': 4}
DEBUG END
F
===================================================
DEBUG START
testreport.nodeid: ::test_fail
names: ['', 'test_fail']
existing_attrs: {}
classnames: ['']
attrs: {'classname': '', 'name': <py._xmlgen.raw object at 0x7f81145761d0>, 'file': '../../../../../../../../../../../../repoX/junit_explore/test_explore.py', 'line': 7}
DEBUG END

===================================================
DEBUG START
testreport.nodeid: ::test_fail
names: ['', 'test_fail']
existing_attrs: {'classname': '', 'name': <py._xmlgen.raw object at 0x7f81145761d0>, 'file': '../../../../../../../../../../../../repoX/junit_explore/test_explore.py', 'line': 7}
classnames: ['']
attrs: {'classname': '', 'name': <py._xmlgen.raw object at 0x7f8114576198>, 'file': '../../../../../../../../../../../../repoX/junit_explore/test_explore.py', 'line': 7}
DEBUG END

Pytest 执行

===================================================
DEBUG START
testreport.nodeid: junit_explore/test_explore.py::test_pass
names: ['junit_explore.test_explore', 'test_pass']
existing_attrs: {}
classnames: ['junit_explore.test_explore']
attrs: {'classname': 'junit_explore.test_explore', 'name': <py._xmlgen.raw object at 0x7f9fd3d28630>, 'file': 'junit_explore/test_explore.py', 'line': 4}
DEBUG END

===================================================
DEBUG START
testreport.nodeid: junit_explore/test_explore.py::test_pass
names: ['junit_explore.test_explore', 'test_pass']
existing_attrs: {'classname': 'junit_explore.test_explore', 'name': <py._xmlgen.raw object at 0x7f9fd3d28630>, 'file': 'junit_explore/test_explore.py', 'line': 4}
classnames: ['junit_explore.test_explore']
attrs: {'classname': 'junit_explore.test_explore', 'name': <py._xmlgen.raw object at 0x7f9fd3d59048>, 'file': 'junit_explore/test_explore.py', 'line': 4}
DEBUG END
F
===================================================
DEBUG START
testreport.nodeid: junit_explore/test_explore.py::test_fail
names: ['junit_explore.test_explore', 'test_fail']
existing_attrs: {}
classnames: ['junit_explore.test_explore']
attrs: {'classname': 'junit_explore.test_explore', 'name': <py._xmlgen.raw object at 0x7f9fd3d51438>, 'file': 'junit_explore/test_explore.py', 'line': 7}
DEBUG END

===================================================
DEBUG START
testreport.nodeid: junit_explore/test_explore.py::test_fail
names: ['junit_explore.test_explore', 'test_fail']
existing_attrs: {'classname': 'junit_explore.test_explore', 'name': <py._xmlgen.raw object at 0x7f9fd3d51438>, 'file': 'junit_explore/test_explore.py', 'line': 7}
classnames: ['junit_explore.test_explore']
attrs: {'classname': 'junit_explore.test_explore', 'name': <py._xmlgen.raw object at 0x7f9fd3d59278>, 'file': 'junit_explore/test_explore.py', 'line': 7}
DEBUG END

这个nodeid一直追踪到item.nodeid

# runner.py
def call_and_report(
    item, when: "Literal['setup', 'call', 'teardown']", log=True, **kwds
):
    call = call_runtest_hook(item, when, **kwds)
    hook = item.ihook
    report = hook.pytest_runtest_makereport(item=item, call=call)

...

def pytest_runtest_makereport(item, call):
    return TestReport.from_item_and_call(item, call) # <--- this item's nodeid

我怀疑 bazel 在整个 bazel 缓存中对符号链接的大量依赖可能会干扰 pytest 以某种方式依赖的这些 filesystem/path 操作。

为什么从 bazel 执行的 pytest 会话将 classname junit 属性留空？

Why does a pytest session executed from bazel leave the classname junit attribute empty?

junit

pytest

bazel