为什么 Nightwatch.js 测试在 CircleCI 上失败了？

Question

我使用 Nightwatch.js 设置了一个简单的 smoketest。他们在本地通过，但他们在 CircleCI 上失败了。

源代码：

const WAIT = 1000;
const NODE_PORT = process.env.NODE_PORT || 3000;

module.exports = {
  'Smoketest' (browser) {
    browser
      .url(`http://localhost:${NODE_PORT}/`)
      .waitForElementVisible('body', WAIT)
      .pause(300)
      .assert.containsText('body', 'Client render')
      .end();
  }
};

本地输出：

> cross-env NODE_PORT=3000 NODE_PATH=source babel-node source/test/e2e.js

Running with config:
{ logDir: '/Users/eric/Dropbox/dev/react-easy-universal/reports',
  reportDir: '/Users/eric/Dropbox/dev/react-easy-universal/reports/test-e2e',
  nightwatchConfig: '/Users/eric/Dropbox/dev/react-easy-universal/nightwatch.json',
  webpackConfig: '/Users/eric/Dropbox/dev/react-easy-universal/webpack.config.js',
  port: '3000',
  server: [Object] }

[Smoketest] Test Suite
======================

Running:  Smoketest
 ✔ Element <body> was visible after 58 milliseconds.
 ✔ Testing if element <body> contains text: "Client render".

OK. 2 assertions passed. (2.831s)

失败的 CircleCI 输出：

REPORT_DIR=${CIRCLE_TEST_REPORTS} LOG_DIR=${CIRCLE_ARTIFACTS} npm run test:e2e -s
Running with config:
{ logDir: '/tmp/circle-artifacts.ddv7oJ3',
  reportDir: '/tmp/circle-junit.hhJgS6N',
  nightwatchConfig: '/home/ubuntu/react-easy-universal/nightwatch.json',
  webpackConfig: '/home/ubuntu/react-easy-universal/webpack.config.js',
  port: '3000',
  server: [Object] }

[Smoketest] Test Suite
======================

Running:  Smoketest
 ✔ Element <body> was visible after 85 milliseconds.
 ✖ Testing if element <body> contains text: "Client render".  - expected "Client render" but got: Untitled
    at Object.Smoketest (smoketest.js:10:15)
    at Module.call (/home/ubuntu/react-easy-universal/node_modules/nightwatch/lib/runner/module.js:63:34)
    at /home/ubuntu/react-easy-universal/node_modules/nightwatch/lib/runner/testcase.js:97:29
    at _fulfilled (/home/ubuntu/react-easy-universal/node_modules/q/q.js:834:54)
    at self.promiseDispatch.done (/home/ubuntu/react-easy-universal/node_modules/q/q.js:863:30)
    at Promise.promise.promiseDispatch (/home/ubuntu/react-easy-universal/node_modules/q/q.js:796:13)
    at /home/ubuntu/react-easy-universal/node_modules/q/q.js:556:49
    at runSingle (/home/ubuntu/react-easy-universal/node_modules/q/q.js:137:13)
    at flush (/home/ubuntu/react-easy-universal/node_modules/q/q.js:125:13)
    at nextTickCallbackWith0Args (node.js:452:9)

FAILED:  1 assertions failed and 1 passed (3.242s)

----------------------------------------------------
TEST FAILURE: 1 assertions failed, 1 passed (3.543s)
 ✖ smoketest
   - Smoketest

Testing if element <body> contains text: "Client render". - Expected "Client render" but got: "Untitled" REPORT_DIR=${CIRCLE_TEST_REPORTS} LOG_DIR=${CIRCLE_ARTIFACTS} npm run test:e2e -s returned exit code 1

最初，浏览器从服务器的一些渲染输出开始。服务器渲染的标题设置为 "Untitled"。然后一些 JavaScript 在客户端加载并运行，客户端渲染发生，将文本更改为 "Client render"。此测试旨在确保客户端渲染按预期工作。

这是nightwatch.json：

{
  "src_folders" : ["source/test/functional/browser"],
  "output_folder" : "./reports/test-e2e",
  "custom_commands_path" : "",
  "custom_assertions_path" : "",
  "page_objects_path" : "",
  "globals_path" : "",

  "selenium" : {
    "start_process" : false,
    "server_path" : "",
    "log_path" : "",
    "host" : "127.0.0.1",
    "port" : 4444,
    "cli_args" : {
      "webdriver.chrome.driver" : "",
      "webdriver.ie.driver" : ""
    }
  },

  "test_settings" : {
    "default" : {
      "launch_url" : "http://localhost",
      "selenium_port"  : 4444,
      "selenium_host"  : "localhost",
      "silent": true,
      "screenshots" : {
        "enabled" : false,
        "path" : ""
      },
      "desiredCapabilities": {
        "browserName": "chrome",
        "javascriptEnabled": true,
        "acceptSslCerts": true
      }
    },

    "chrome" : {
      "desiredCapabilities": {
        "browserName": "chrome",
        "javascriptEnabled": true,
        "acceptSslCerts": true
      }
    }
  }
}

我怀疑这可能是一个竞争条件，认为断言在 CircleCI 有机会加载和处理 JavaScript 之前运行，所以我尝试在断言之前插入一个暂停。

如有任何想法，我们将不胜感激。

Answer 1

我也有这个问题，看起来（仍然需要重新检查）我找到了解决方案。

确保你在每个测试中都有最后的代码。

browser.end();

为什么 Nightwatch.js 测试在 CircleCI 上失败了？

Why are Nightwatch.js tests failing on CircleCI?

circleci

nightwatch.js