为什么 SauceLabs 在我的 QUnit 测试明显通过时却说它失败了？

Question

我们正在尝试使用 SauceLabs 来验证我们基于浏览器的 QUnit 测试是否通过了流行的 device/browser 组合...

测试通过当我们在浏览器中查看它们时：https://ordenado.herokuapp.com/

但出于某种原因，SauceLabs 告诉我们他们 "failed"...

参见：https://saucelabs.com/tests/5b0f07813a7f4934bb44b07606ea2fd5

使用的 cURL 命令

作为参考，我们使用了以下 curl 命令：

curl https://saucelabs.com/rest/v1/ordem/js-tests \
-X POST \
-u ordem:SECRET_KEY \
-H 'Content-Type: application/json' \
--data '{
    "platforms": [
      ["Windows 8.1", "internet explorer", "11"],
      ["Windows 8", "internet explorer", "10"],
      ["Windows 8.1", "firefox", "beta"],
      ["Windows 8", "firefox", "37"],
      ["Windows 7", "firefox", "32"],
      ["OS X 10.8", "safari", "6"],
      ["OS X 10.8", "chrome", "37"],
      ["Linux", "chrome", "30"],
      ["Linux", "firefox", "dev"],
      ["OS X 10.10","iphone", "7.0"],
      ["OS X 10.10","iphone", "8.2"],
      ["OS X 10.10","ipad", "7.0"],
      ["OS X 10.10","ipad", "8.2"]
    ],
    "url": "https://qunit.herokuapp.com/test/test.html?coverage=true",
    "framework": "qunit",
    "name":"ordem",
    "public": "public",
    "build": "build-007"
}'

另外，有没有其他人遇到过以下错误：

"The Sauce VMs failed to start the browser or device"

访问：https://saucelabs.com/u/ordem 获取完整的测试列表。单击 Sauce 声称的任何一个 "failed" 并观看视频以查看测试通过！！

非常感谢任何见解！

Answer 1

您应该添加几个挂钩来报告来自 SauceLabs 的 qunit 测试结果。

有 an article how to get started with qunit. There mentioned repository with example 以下代码段：

var log = [];
QUnit.done = function (test_results) {
  var tests = log.map(function(details){
    return {
      name: details.name,
      result: details.result,
      expected: details.expected,
      actual: details.actual,
      source: details.source
    }
  });
  test_results.tests = tests;

  // delaying results a bit cause in real-world
  // scenario you won't get them immediately
  setTimeout(function () { window.global_test_results = test_results; }, 2000);
};
QUnit.testStart(function(testDetails){
  QUnit.log = function(details){
    if (!details.result) {
     details.name = testDetails.name;
     log.push(details);
    }
 }
});

在您的测试和结果正确报告之前添加此代码

为什么 SauceLabs 在我的 QUnit 测试明显通过时却说它失败了？

Why does SauceLabs say my QUnit Test Fails when its clearly passing?

javascript

unit-testing

testing

qunit

saucelabs

使用的 cURL 命令