'Inconclusive' 项测试被 Azure DevOps 报告为 'not run'
'Inconclusive' tests are reported as 'not run' by Azure DevOps
我有一些针对外部服务的集成测试，有时可能会失败。对于这些，我 运行ning 'Assert.Inconclusive("errMsgHere");' 这样测试就不会导致整个 CI/CD 管道失败；我只是想要一个警告。虽然当我 运行 在 Visual Studio 中进行本地测试时，这确实会给出一个警告图标，但对于 Azure DevOps，测试报告为 'not run'。此外，在管道 运行 的 'Tests' 选项卡中，测试的调试详细信息有一个错误消息的空白条目而不是“errMsgHere”，即使该错误消息包含在输出的 .trx 文件中！当然，Azure DevOps 应该显示 Assert.Inconclusive() 输出的消息。有什么方法可以获取此消息，或者将测试结果报告为比“not 运行”更有用的东西？也许 'warning' 或 'inconclusive'?
根据此 response from Sarabjot Singh on the github issue 对于此任务，此行为似乎是设计使然。
@osolo The Inconclusive tests are treated the same as skipped tests.
The behavior for not reporting them as failed and failing the task is
by design currently. It is based on the precedence set by the test
platform (https://github.com/microsoft/vstest) and the test framework
(https://github.com/microsoft/testfx)
What is the behavior when you run the tests locally on your machine
using the vstest.console runner? You can generate the trx result file
(using --logger:trx option) and then compare it what is being shown in
the pipeline UI.
此外，在开发社区提交的 linked bug 似乎不太可能得到解决：
I have detected that for the last 35 days, this issue didn't have much
product team activity and a very small amount of new votes or
comments. Based on this, its severity, and affected area, it’s my
experience that this issue is very unlikely to be fixed.
似乎确实提到了一种解决方法，尽管它可能达不到您想要的结果。我知道这可能不是您想要的，但是您可以通过为测试任务设置“Continue on error”选项将测试失败视为警告。不过，它会将所有失败都视为警告，所以也许有些情况下您确实想要停止管道对您的情况没有用。
我有一些针对外部服务的集成测试，有时可能会失败。对于这些，我 运行ning 'Assert.Inconclusive("errMsgHere");' 这样测试就不会导致整个 CI/CD 管道失败；我只是想要一个警告。虽然当我 运行 在 Visual Studio 中进行本地测试时，这确实会给出一个警告图标，但对于 Azure DevOps，测试报告为 'not run'。此外，在管道 运行 的 'Tests' 选项卡中，测试的调试详细信息有一个错误消息的空白条目而不是“errMsgHere”，即使该错误消息包含在输出的 .trx 文件中！当然，Azure DevOps 应该显示 Assert.Inconclusive() 输出的消息。有什么方法可以获取此消息，或者将测试结果报告为比“not 运行”更有用的东西？也许 'warning' 或 'inconclusive'?
根据此 response from Sarabjot Singh on the github issue 对于此任务，此行为似乎是设计使然。
@osolo The Inconclusive tests are treated the same as skipped tests.
The behavior for not reporting them as failed and failing the task is by design currently. It is based on the precedence set by the test platform (https://github.com/microsoft/vstest) and the test framework (https://github.com/microsoft/testfx)
What is the behavior when you run the tests locally on your machine using the vstest.console runner? You can generate the trx result file (using --logger:trx option) and then compare it what is being shown in the pipeline UI.
此外，在开发社区提交的 linked bug 似乎不太可能得到解决：
I have detected that for the last 35 days, this issue didn't have much product team activity and a very small amount of new votes or comments. Based on this, its severity, and affected area, it’s my experience that this issue is very unlikely to be fixed.
似乎确实提到了一种解决方法，尽管它可能达不到您想要的结果。我知道这可能不是您想要的，但是您可以通过为测试任务设置“Continue on error”选项将测试失败视为警告。不过，它会将所有失败都视为警告，所以也许有些情况下您确实想要停止管道对您的情况没有用。