来自 wdio cucumber json 记者的 Jira Xray 中的错误测试结果

Question

我们有一个链接到 Cucumber 测试的故事，我们在测试执行中执行此测试。测试执行中的功能文件如下所示：

@ABC-164
@ABC-163
Feature: Homepage test
    @ABC-162
    Scenario: Homepage test
        Feature: Go to homepage
            Scenario: Homepage
                Given I open the url "https://example.com/"
                Then I expect that the url is "https://google.com"

第一个总是通过，第二个不通过。我们正在生成来自 wdio cucumber json 报告者的 JSON 报告，您可以在下面看到：

[
  {
    "keyword": "Feature",
    "type": "feature",
    "description": "",
    "line": 3,
    "name": "Homepage test",
    "uri": "Can not be determined",
    "tags": [
      {
        "type": "Tag",
        "location": { "line": 1, "column": 1 },
        "name": "@ABC-164"
      },
      {
        "type": "Tag",
        "location": { "line": 2, "column": 1 },
        "name": "@ABC-163"
      }
    ],
    "elements": [
      {
        "keyword": "Scenario",
        "type": "scenario",
        "description": "\t\tFeature: Go to homepage",
        "name": "Homepage test",
        "tags": [
          { "name": "@ABC-164", "location": { "line": 1, "column": 1 } },
          { "name": "@ABC-163", "location": { "line": 2, "column": 1 } },
          { "name": "@ABC-162", "location": { "line": 6, "column": 2 } }
        ],
        "id": "homepage-test;homepage-test",
        "steps": [
          {
            "arguments": [],
            "keyword": "Before",
            "name": "Hook",
            "result": { "status": "passed", "duration": 2000000 },
            "line": "",
            "match": { "location": "can not be determined with webdriver.io" }
          },
          {
            "arguments": [],
            "keyword": "Before",
            "name": "Hook",
            "result": { "status": "passed", "duration": 0 },
            "line": "",
            "match": { "location": "can not be determined with webdriver.io" }
          }
        ]
      },
      {
        "keyword": "Scenario",
        "type": "scenario",
        "description": "",
        "name": "Homepage",
        "tags": [
          { "name": "@ABC-164", "location": { "line": 1, "column": 1 } },
          { "name": "@ABC-163", "location": { "line": 2, "column": 1 } }
        ],
        "id": "homepage-test;homepage",
        "steps": [
          {
            "arguments": [],
            "keyword": "Before",
            "name": "Hook",
            "result": { "status": "passed", "duration": 0 },
            "line": "",
            "match": { "location": "can not be determined with webdriver.io" }
          },
          {
            "arguments": [],
            "keyword": "Given",
            "name": "I open the url \"https://example.com/\"",
            "result": { "status": "passed", "duration": 587000000 },
            "line": 12,
            "match": { "location": "can not be determined with webdriver.io" }
          },
          {
            "arguments": [],
            "keyword": "Then",
            "name": "I expect that the url is \"https://google.com\"",
            "result": {
              "status": "failed",
              "duration": 11000000,
              "error_message": "Error: expect(received).toEqual(expected)"
            },
            "line": 13,
            "match": { "location": "can not be determined with webdriver.io" }
          },
          {
            "arguments": [],
            "keyword": "After",
            "name": "Hook",
            "result": { "status": "passed", "duration": 0 },
            "line": "",
            "match": { "location": "can not be determined with webdriver.io" }
          }
        ]
      }
    ],
    "id": "homepage-test",
    "metadata": {
      "browser": { "name": "chrome", "version": "88.0.4324.150" },
      "device": "Device name not known",
      "platform": { "name": "osx", "version": "Version not known" }
    }
  }
]

Xray 仅从第一个对象的 Before Hooks 中读取状态，完全忽略其余状态。

当我们使用 Jenkins 插件以及通过 curl 直接发送时会出现问题：curl -k -H "Content-Type: application/json" -X POST -u login:pass --data @result.json https://jira/rest/...

如有任何帮助，我们将不胜感激。

Answer 1

场景下不能有功能。这是一个语法错误，尽管 Cucumber 可能无法正确报告它。

Answer 2

正如 Aslak 在评论中提到的，.feature 文件语法不正确。一个合适的 .feature 文件就像前面的例子一样。还请记住，您拥有的标签 (@ABC-xxx) 可能需要一些前缀，例如 REQ 或 TEST，具体取决于您的 settings.

顺便说一句，不清楚您是否从 Xray 导出测试执行以生成 .feature 文件，但我认为情况并非如此。只是为了强调，为了根据现有测试（Cucumber Scenario/Scenario 大纲）报告结果，它们必须事先存在于 Xray 中。

@ABC-164
@ABC-163
Feature: Homepage test
    @ABC-162
    Scenario: Homepage
      Given I open the url "https://example.com/"
      Then I expect that the url is "https://google.com"*

Answer 3

对于那些会遇到同样问题的人。我们设法找到了解决方案。非常感谢您的回答！

看起来我们在 Jira 中有一个旧的特征文件结构（那时候我们没有故事和先决条件）。当我们开始正确使用它时：将故事链接到测试并添加前提条件，然后 Xray 会创建双倍的场景部分。所以我们从 Cucumber Tests 中删除了多余的部分，创建了适当的前提条件，现在特征文件就这样正确地创建了：

@ABC-169
@ABC-131
Feature: Main feature title

    Background:
        #@ABC-166
        Given I open the url "https://example.com/"
        Then I expect that the url is "https://google.com"

    @ABC-132 @ABC-168 @ABC-141
    Scenario: Main scenario
        Given Lorem ipsum
        Then Et sit amet

来自 wdio cucumber json 记者的 Jira Xray 中的错误测试结果

Wrong tests results in Jira Xray from wdio cucumber json reporter

json

cucumber

webdriver-io

jira-xray