Nightwatch waitForElementVisible - abortOnFailure 参数设置为 false - 测试退出状态非零

Question

我正在使用 Nightwatch API docs 中的 waitForElementVisible(<selector>, <timeout>, false) 命令，但它的行为并不像我预期的那样。我如何调整此代码以获得预期的行为？

预期行为：

call .waitForElementVisible('foobar', 10, false)

see command fail and continue execution with the next command

all other commands pass

see exit status of 0 from script

实际行为：

call .waitForElementVisible('foobar', 10, false)

see command fail and continue execution with the next command

all other commands pass

see exit status of 1 from script

这里是重现的示例代码

module.exports = {
  tags: ['smoke'],

  before: browser =>
    browser
      .maximizeWindow('current').url('https://google.com'),

  after: browser => browser.end(),

  'smoke test': browser =>
    browser
      .waitForElementVisible('foobar', 10, false)
      .waitForElementVisible('img')
      .assert.visible('img'),
};

这是运行命令的控制台输出：

Starting selenium server in parallel mode... started - PID:  75459

Started child process for: 01_smoke 
 01_smoke   \n
 01_smoke   [01 Smoke] Test Suite
=========================
 01_smoke   
 01_smoke   Results for:  smoke test
 01_smoke   ✖ Timed out while waiting for element <foobar> to be present for 10 milliseconds.  - expected "visible" but got: "not found"
 01_smoke       at Object.smokeTest [as smoke test] (/path/to/tests/01_smoke.js:12:8)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
 01_smoke   ✔ Element <img> was visible after 33 milliseconds.
 01_smoke   ✔ Testing if element <img> is visible.
 01_smoke   
 01_smoke   Retrying (1/3):  smoke test
 01_smoke   ✖ Timed out while waiting for element <foobar> to be present for 10 milliseconds.  - expected "visible" but got: "not found"
 01_smoke       at Object.smokeTest [as smoke test] (/path/to/tests/01_smoke.js:12:8)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
 01_smoke   ✔ Element <img> was visible after 21 milliseconds.
 01_smoke   ✔ Testing if element <img> is visible.
 01_smoke   
 01_smoke   Retrying (2/3):  smoke test
 01_smoke   ✖ Timed out while waiting for element <foobar> to be present for 10 milliseconds.  - expected "visible" but got: "not found"
 01_smoke       at Object.smokeTest [as smoke test] (/path/to/tests/01_smoke.js:12:8)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
 01_smoke   ✔ Element <img> was visible after 20 milliseconds.
 01_smoke   ✔ Testing if element <img> is visible.
 01_smoke   Retrying (3/3):  smoke test
 01_smoke   ✖ Timed out while waiting for element <foobar> to be present for 10 milliseconds.  - expected "visible" but got: "not found"
 01_smoke       at Object.smokeTest [as smoke test] (/path/to/tests/01_smoke.js:12:8)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
 01_smoke   ✔ Element <img> was visible after 20 milliseconds.
 01_smoke   ✔ Testing if element <img> is visible.
 01_smoke   FAILED:  1 assertions failed and 2 passed (53ms)

  >> 01_smoke finished.  


 _________________________________________________

 TEST FAILURE:  1 assertions failed, 2 passed. (6.259s)

 ✖ 01_smoke

   - smoke test (53ms)
   Timed out while waiting for element <foobar> to be present for 10 milliseconds.  - expected "visible" but got: "not found"
       at Object.smokeTest [as smoke test] (/path/to/tests/01_smoke.js:12:8)
       at _combinedTickCallback (internal/process/next_tick.js:131:7)

Answer 1

是的，它应该是这样的！我认为您误解了 abortOnFailure 标志对 waitForVisible 命令的作用方式。 false 标志仅向方法提供字符 以供测试运行器评估为不间断步骤。但这并不意味着它不会将该步骤计为失败步骤。

注意：类似的事情发生在assert/verify的情况下（其中verify是一个不间断的断言，类似于waitForElementVisible).

的 abortOnFailure: false 参数

虽然我知道人们会从哪里得到这种印象。如果您阅读了 API 调用的描述，它说：

If the element fails to be present and visible in the specified amount of time, the test fails. You can change this by setting abortOnFailure to false.

这让您认为也许测试最终会通过，即使 waitForVisible 命令失败了。 但是... API 调用的 Parameters 部分可以帮助我们，删除错误假设：

By the default if the element is not found the test will fail. Set this to false if you wish for the test to continue even if the assertion fails. To set this globally you can define a property abortOnAssertionFailure in your globals.

最后... DOC 可能会让您失望，代码永远不会说谎：

  process.on('exit', function (code) {
    var exitCode = code;

    if (exitCode === 0 && globalResults && (globalResults.errors > 0 || globalResults.failed > 0)) {
      exitCode = 1;
    }

    process.exit(exitCode);
  });

以上是 Nightwatch 的测试运行程序 (nightwatch/lib/runner/run.js) 的片段。自己看看它认为什么是有效的 exit code 1 条件。

干杯！

Nightwatch waitForElementVisible - abortOnFailure 参数设置为 false - 测试退出状态非零

Nightwatch waitForElementVisible - abortOnFailure parameter set to false - test exit status non-zero

nightwatch.js