Python BDD 工具 "behave" 的 Bazel 测试规则

Question

问题：

你好，亲爱的 bazel 英雄们！

对于我们的系统测试，我正在尝试使用 bazel as

将行为驱动开发工具 behave 升级为运行

bazel test system/acceptance_criteria:nice

需要完成的事情：

调用命令行 behave <feature_file>（通过过滤运行文件并使用 <feature_directory> 而不是 <feature_file> 解决）
将文件夹 steps 添加到运行文件（通过在文件前明确添加路径前缀解决）
将相关的 Python 步骤文件添加到运行文件（已解决）
将被测系统添加到 PYTHONPATH（打开）

您有什么建议或您对如何解决此问题有任何其他建议？

使用 Skylark 制定新规则（我已经试过了，但在 PYTHONPATH 和运行文件中苦苦挣扎。我得到了 ModuleNotFoundError: No module named 'my_system'）
将 Skylark 用于带有 genrule 的宏
在 py_test 中调用脚本（可能但不太灵活的解决方法，因为我没有弄清楚如何将参数从 bazel 传递到脚本）
一些其他最佳实践？

干杯，克里斯

如果可能，请提供一个最小示例来重现问题：

我尝试的几乎是在 Skylark 中复制 py_test 的行为并让它运行命令 behave <feature_directory> 或者 python -m behave <feature_directory>:

project/

WORKSPACE（空）
system/
- BUILD
- my_system.py
- acceptance_criteria/
  - BUILD
  - nice.feature
  - steps/
    - reusable_steps.py
tools/
- BUILD（空）
- behave_rule.bzl
- bazel.rc

system/BUILD:

py_binary(
    name = "main",
    main = "my_system.py",
    srcs = ["my_system.py"],
    # deps = [], excluded for this example
    imports = ["."], # needed for adding PYTHONPATH in acceptance_criteria
    visibility = ["//system/acceptance_criteria:__pkg__"]
)

system/my_system.py:

class MySystem():
    def yeah(self):
        return "yeah it works"

system/acceptance_criteria/BUILD:

load("//tools:behave_rule.bzl", "py_bdd_test")

py_bdd_test(
    name = "nice",
    feats = [
        "nice.feature",
    ],
    steps = [
        "steps/reusable_steps.py",
    ],
    deps = [
        # TODO figure out how to add the implicit imports to the PYTHONPATH
        "//system:main",
    ],
    size = "small",
)

system/acceptance_criteria/nice.feature:

Feature: Multiprocess software

Scenario: 4 processes
  Given the device is powered on

steps/reusable_steps.py:

from behave import *
from my_system import MySystem

@Given("the device is powered on")
def step_impl(context):
    raise NotImplementedError(MySystem().yeah())

tools/behave_rule.bzl:

# =============================================================================
# Description: Adds a test rule for the BDD tool behave to the bazel rule set.
# Knowledge:
# * https://bazel.build/versions/master/docs/skylark/cookbook.html
# * https://bazel.build/versions/master/docs/skylark/rules.html
# * https://bazel.build/versions/master/docs/skylark/lib/ctx.html
# * http://pythonhosted.org/behave/gherkin.html
# =============================================================================

"""Private implementation of the rule py_bdd_test.
"""
def _rule_implementation(ctx):

  # Store the path of the first feature file
  features_dir = ctx.files.feats[0].dirname

  # We want a test target so make it create an executable output.
  # https://bazel.build/versions/master/docs/skylark/rules.html#test-rules
  ctx.file_action(
      # Access the executable output file using ctx.outputs.executable.
      output=ctx.outputs.executable,
      content="behave %s" % features_dir,
      executable=True
  )
  # The executable output is added automatically to this target.

  # Add the feature and step files for behave to the runfiles.
  # https://bazel.build/versions/master/docs/skylark/rules.html#runfiles
  return [DefaultInfo(
      # Create runfiles from the files specified in the data attribute.
      # The shell executable - the output of this rule - can use them at runtime.
      # It is also possible to define data_runfiles and default_runfiles.
      # However if runfiles is specified it's not possible to define the above
      # ones since runfiles sets them both.
      runfiles = ctx.runfiles(
          files = ctx.files.feats + ctx.files.steps + ctx.files.deps)
  )]

"""An example documentation.

Args:
  name:
    A unique name for this rule.
  feats:
    Feature files used to run this target.
  steps:
    Files containing the mapping of feature steps to actual system API calls.
    Note: Since this rule implicitely uses the BDD tool "behave" they must
be in the "steps" folder (https://pythonhosted.org/behave/gherkin.html).
  deps:
    System to test.
"""
py_bdd_test = rule(
    implementation=_rule_implementation,
    attrs={
      # Do not declare "name": It is added automatically.
      "feats": attr.label_list(allow_files=True),
      "steps": attr.label_list(allow_files=True),
      "deps":
        attr.label_list(
            mandatory=True,
            non_empty=True,)
    },
    test=True,
)

tools/bazel.rc:

test --test_output=errors

环境信息

操作系统：Windows 10 with docker（参见下面的 Dockerfile）
Bazel 版本（bazel info release 的输出）：发布 0.5.1rc1

Dockerfile:

FROM ubuntu:xenial

# Install essentials
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update \
  && apt-get install -y gnupg git \
  && apt-get clean

# -----------------------------------------------------------------------------

# Install the awesome build automation tool bazel
# https://bazel.build/
# version > 0.4.5 since it has a bug in the rule extension skylark
RUN echo "deb http://storage.googleapis.com/bazel-apt testing jdk1.8" \
  > /etc/apt/sources.list.d/bazel.list
RUN apt-key adv --keyserver pool.sks-keyservers.net --recv-key 3D5919B448457EE0
RUN apt-get update \
  && apt-get install -y openjdk-8-jdk bazel \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/* && mkdir /var/lib/apt/lists/partial
ENV DEBIAN_FRONTEND ""

# Setup environment for bazel
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64

# Run bazel a first time for it to self-extract
RUN /usr/bin/bazel version

# -----------------------------------------------------------------------------

# Install Python (using the non-default version 3.6)
RUN echo "deb http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial main" \
  > /etc/apt/sources.list.d/jonathonf-ubuntu-python-3_6-xenial.list \
  && apt-key adv --keyserver pool.sks-keyservers.net --recv-key 8CF63AD3F06FC659 \
  && apt-get update \
  && apt-get install -y python3.6

# Create symlink from python3.6 to python
RUN ln -s /usr/bin/python3.6 /usr/local/bin/python

# Install the Python package manager pip (for the non-default version 3.6)
# https://en.wikipedia.org/wiki/Pip_(package_manager)
RUN apt-get update \
  && apt-get install -y wget \
  && wget https://bootstrap.pypa.io/get-pip.py \
  && python3.6 get-pip.py \
  && rm get-pip.py \
  && pip install --upgrade pip

# -----------------------------------------------------------------------------

# Install the Gherkin-based BDD tool "behave" for Python
RUN pip install behave

# -----------------------------------------------------------------------------

# Our build environment is based on bazel. Now run tests with it.
ENTRYPOINT "/usr/bin/bazel"

您是否通过网络搜索找到任何相关内容？

不幸的是没有。正如 behave_rule.bzl 的评论中提到的，我调查了

https://bazel.build/versions/master/docs/skylark/cookbook.html
https://bazel.build/versions/master/docs/skylark/rules.html
https://bazel.build/versions/master/docs/skylark/lib/ctx.html

我也找到了

https://github.com/bazelbuild/bazel/issues/702

还有什么有用的信息、日志或输出吗？

命令行输出：

root@eec1fa791491:/project# bazel test system/acceptance_criteria:nice
INFO: Found 1 test target...
FAIL: //system/acceptance_criteria:nice (see /root/.cache/bazel/_bazel_root/2ca1f4ebdc59348ffdc31d97a51a98d5/execroot/project/bazel-out/local-fastbuild/testlogs/system/acceptance_criteria/nice/test.log).
INFO: From Testing //system/acceptance_criteria:nice:
==================== Test output for //system/acceptance_criteria:nice:
Exception ModuleNotFoundError: No module named 'my_system'
Traceback (most recent call last):
  File "/usr/local/bin/behave", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/dist-packages/behave/__main__.py", line 109, in main
    failed = runner.run()
  File "/usr/local/lib/python3.6/dist-packages/behave/runner.py", line 672, in run
return self.run_with_paths()
File "/usr/local/lib/python3.6/dist-packages/behave/runner.py", line 678, in run_with_paths
    self.load_step_definitions()
File "/usr/local/lib/python3.6/dist-packages/behave/runner.py", line 658, in load_step_definitions
    exec_file(os.path.join(path, name), step_module_globals)
File "/usr/local/lib/python3.6/dist-packages/behave/runner.py", line 304, in exec_file
    exec(code, globals, locals)
File "system/acceptance_criteria/steps/reusable_steps.py", line 5, in <module>
    from my_system import MySystem
ModuleNotFoundError: No module named 'my_system'
================================================================================
Target //system/acceptance_criteria:nice up-to-date:
bazel-bin/system/acceptance_criteria/nice
INFO: Elapsed time: 0.538s, Critical Path: 0.22s
//system/acceptance_criteria:nice                                        FAILED in 0.2s
/root/.cache/bazel/_bazel_root/2ca1f4ebdc59348ffdc31d97a51a98d5/execroot/project/bazel-out/local-fastbuild/testlogs/system/acceptance_criteria/nice/test.log

Executed 1 out of 1 test: 1 fails locally.

来自：

https://github.com/bazelbuild/bazel/issues/3194#issuecomment-309236573

Answer 1

几年过去了，坐下来想出了这个

bdd.bzl

"""Custom Bazel Starlark rule for running behave tests.

Idea taken from https://github.com/bazelbuild/examples/
Also read https://docs.bazel.build/versions/main/skylark/concepts.html
"""

def _bdd_test_impl(ctx):
    # Assemble the command to run.
    bdd_tool = 'behave'
    default_args = '--no-capture --no-capture-stderr --no-logcapture'
    # Optional arguments are passed either
    #  explicitly as --test_arg=--foo="bar bar" (no space in between)
    #  or implicitly via ctx.attr.args
    optional_args = '"$@"'

    feature_file = ctx.file.main.basename
    features_path = ctx.label.package

    command = ' '.join([bdd_tool, default_args, features_path, '--include', feature_file])
    command = ' '.join([
        bdd_tool, features_path, '--include', feature_file, default_args, optional_args
    ])

    # Wrap an executable around the command.
    ctx.actions.write(
        output = ctx.outputs.executable,
        content = command,
    )

    # Ensure the files needed by the command are available at runtime
    srcs = ctx.files.main + ctx.files.steps + ctx.files.data
    deps = []
    for dep in ctx.attr.deps:
        # Collecting the files from the dependency tree is a little more hassle
        # since not well-documented and prone to change between bazel versions.
        deps += dep.default_runfiles.files.to_list()

    return [DefaultInfo(
        runfiles = ctx.runfiles(
            files = srcs + deps,
        ),
    )]

bdd_test = rule(
    implementation = _bdd_test_impl,
    test = True,
    attrs = {
        "main": attr.label(
            allow_single_file = [".feature"],
            mandatory = True,
        ),
        "steps": attr.label_list(
            allow_files = True,
        ),
        "deps": attr.label_list(),
        "data": attr.label_list(
            allow_files = True,
        ),
    },
)

BUILD

load("bdd.bzl", "bdd_test")

[bdd_test(
    name = filename[:filename.rfind(".feature")],
    main = filename,
    steps = ["environment.py"] + glob(["steps/*.py"]),
    deps = [
        # ':some_fixture',
    ],
) for filename in glob(["*.feature"])]

Python BDD 工具 "behave" 的 Bazel 测试规则

Bazel test rule for the Python BDD tool "behave"

python

bdd

pythonpath

bazel

问题：

如果可能，请提供一个最小示例来重现问题：

环境信息

您是否通过网络搜索找到任何相关内容？

还有什么有用的信息、日志或输出吗？