Nightwatch waitForElementVisible - abortOnFailure参数设置为false - 测试退出状态非零

Question

我正在使用来自waitForElementVisible(<selector>, <timeout>, false)的Nightwatch API docs命令，它的表现并不像我预期的那样。我如何调整此代码以获得预期的行为？

预期行为：

• 打电话给.waitForElementVisible('foobar', 10, false)
• 看命令失败并继续执行下一个命令
• 所有其他命令都通过
• 请参阅脚本中的退出状态0

实际行为：

• 打电话给.waitForElementVisible('foobar', 10, false)
• 看命令失败并继续执行下一个命令
• 所有其他命令都通过
• 请参阅脚本中的退出状态1

这是重现的示例代码

module.exports = {
  tags: ['smoke'],

  before: browser =>
    browser
      .maximizeWindow('current').url('https://google.com'),

  after: browser => browser.end(),

  'smoke test': browser =>
    browser
      .waitForElementVisible('foobar', 10, false)
      .waitForElementVisible('img')
      .assert.visible('img'),
};

这是运行该命令的控制台输出：

Starting selenium server in parallel mode... started - PID:  75459

Started child process for: 01_smoke 
 01_smoke   \n
 01_smoke   [01 Smoke] Test Suite
=========================
 01_smoke   
 01_smoke   Results for:  smoke test
 01_smoke   ✖ Timed out while waiting for element <foobar> to be present for 10 milliseconds.  - expected "visible" but got: "not found"
 01_smoke       at Object.smokeTest [as smoke test] (/path/to/tests/01_smoke.js:12:8)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
 01_smoke   ✔ Element <img> was visible after 33 milliseconds.
 01_smoke   ✔ Testing if element <img> is visible.
 01_smoke   
 01_smoke   Retrying (1/3):  smoke test
 01_smoke   ✖ Timed out while waiting for element <foobar> to be present for 10 milliseconds.  - expected "visible" but got: "not found"
 01_smoke       at Object.smokeTest [as smoke test] (/path/to/tests/01_smoke.js:12:8)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
 01_smoke   ✔ Element <img> was visible after 21 milliseconds.
 01_smoke   ✔ Testing if element <img> is visible.
 01_smoke   
 01_smoke   Retrying (2/3):  smoke test
 01_smoke   ✖ Timed out while waiting for element <foobar> to be present for 10 milliseconds.  - expected "visible" but got: "not found"
 01_smoke       at Object.smokeTest [as smoke test] (/path/to/tests/01_smoke.js:12:8)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
 01_smoke   ✔ Element <img> was visible after 20 milliseconds.
 01_smoke   ✔ Testing if element <img> is visible.
 01_smoke   Retrying (3/3):  smoke test
 01_smoke   ✖ Timed out while waiting for element <foobar> to be present for 10 milliseconds.  - expected "visible" but got: "not found"
 01_smoke       at Object.smokeTest [as smoke test] (/path/to/tests/01_smoke.js:12:8)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
 01_smoke   ✔ Element <img> was visible after 20 milliseconds.
 01_smoke   ✔ Testing if element <img> is visible.
 01_smoke   FAILED:  1 assertions failed and 2 passed (53ms)

  >> 01_smoke finished.  


 _________________________________________________

 TEST FAILURE:  1 assertions failed, 2 passed. (6.259s)

 ✖ 01_smoke

   - smoke test (53ms)
   Timed out while waiting for element <foobar> to be present for 10 milliseconds.  - expected "visible" but got: "not found"
       at Object.smokeTest [as smoke test] (/path/to/tests/01_smoke.js:12:8)
       at _combinedTickCallback (internal/process/next_tick.js:131:7)

Answer 1

是的，这就是应该的样子！我认为你误解了abortOnFailure旗为waitForVisible命令工作的方式。 false标志仅为测试运行者评估的角色提供方法作为非破坏步骤。但这并不意味着它不会将该步骤视为失败的一步。

注意：类似的事情发生在assert/verify案例中（其中verify是一个非破坏性断言，类似于abortOnFailure: false的waitForElementVisible参数）。

---

我可以看到一个人会得到那种印象的地方。如果你阅读了API调用的描述，它说：

如果元素在指定的时间内不存在且可见，则测试失败。您可以通过将abortOnFailure设置为false来更改此设置。

这让你觉得即使waitForVisible命令失败，也许测试最终会通过。但是...... API调用的Parameters部分对我们有帮助，删除了错误的假设：

默认情况下，如果未找到该元素，则测试将失败。如果您希望测试继续，即使断言失败，也将此设置为false。要全局设置，可以在globals中定义属性abortOnAssertionFailure。

---

最后...... DOC可能会让您失败，代码绝不会撒谎：

  process.on('exit', function (code) {
    var exitCode = code;

    if (exitCode === 0 && globalResults && (globalResults.errors > 0 || globalResults.failed > 0)) {
      exitCode = 1;
    }

    process.exit(exitCode);
  });

上面是一个片段Nightwatch的测试跑者（nightwatch/lib/runner/run.js）。看看它认为有效的exit code 1条件。

干杯!