Azure DevOps Server:为什么 Visual Studio 测试步骤需要这么长时间才能进行 运行 单元测试?
Azure DevOps Server: Why does a Visual Studio Test step take so long to run unit tests?
在 DLL 中的 Azure DevOps Server build system installed on premises, I have a Build Pipeline with a Visual Studio Test step. The step is configured to discover and run XUnit 单元测试中。
步骤是通过 GUI 定义的。这是它的属性:
Version: 2.*
Select tests using: Test assemblies
Test files:
**\*test*.dll
!**\*TestAdapter.dll
!**\obj\**
Search folder: $(System.DefaultWorkingDirectory)
Select test platform using: Version
Test platform version: Latest
步骤 运行s 并成功通过,确定了 18 个测试。但是,这样做花费了不合理的时间:~51 秒。
运行 日志显示它在启动 vstest.console.exe
之前花费了大约 40 秒,似乎什么也没做。请在下面的完整日志中查看此行:
2019-09-23T07:11:35.7961563Z ========================================================
我的问题 - 为什么 Visual Studio 测试步骤会花费这么多时间空闲?我该怎么做才能减少它浪费的时间?
更新 1:作为实验,我创建了一个新的 git 存储库,其中仅包含我的解决方案构建的 (~100) 个 DLL。我创建了一个新管道,它只提取这些 DLL 和 运行s 单元测试。当我 运行 这个时,它 运行 快多了。特别是日志中原来需要 40 秒的两行现在只用了不到一秒。
更新 2:我试图通过指定测试 DLL 的完整路径并从 'Test files' 中删除所有 **
模式来加快速度 属性。这一步仍然 运行 和以前一样慢。
2019-09-23T07:11:32.4121152Z ##[section]Starting: run unit tests
2019-09-23T07:11:32.4135871Z ==============================================================================
2019-09-23T07:11:32.4136477Z Task : Visual Studio Test
2019-09-23T07:11:32.4136755Z Description : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2019-09-23T07:11:32.4137066Z Version : 2.143.8
2019-09-23T07:11:32.4137295Z Author : Microsoft Corporation
2019-09-23T07:11:32.4137526Z Help : [More information](https://go.microsoft.com/fwlink/?LinkId=835764)
2019-09-23T07:11:32.4138042Z ==============================================================================
2019-09-23T07:11:35.0894195Z SystemVssConnection exists true
2019-09-23T07:11:35.0937440Z SystemVssConnection exists true
2019-09-23T07:11:35.2406351Z Running tests using vstest.console.exe runner.
2019-09-23T07:11:35.2407337Z ======================================================
2019-09-23T07:11:35.2408408Z Test selector : Test assemblies
2019-09-23T07:11:35.2409125Z Test filter criteria : null
2019-09-23T07:11:35.2409694Z Search folder : C:\pool\_work8\s
2019-09-23T07:11:35.2412135Z VisualStudio version selected for test execution : latest
2019-09-23T07:11:35.2758339Z Run in parallel : false
2019-09-23T07:11:35.2810059Z Run in isolation : false
2019-09-23T07:11:35.2813902Z Path to custom adapters : null
2019-09-23T07:11:35.2831140Z Other console options : null
2019-09-23T07:11:35.2883526Z Code coverage enabled : false
2019-09-23T07:11:35.2884892Z Diagnostics enabled : false
2019-09-23T07:11:35.2937817Z SystemVssConnection exists true
2019-09-23T07:11:35.3039161Z Run the tests locally using vstest.console.exe
2019-09-23T07:11:35.3040102Z ========================================================
2019-09-23T07:11:35.3075806Z Test selector : Test assemblies
2019-09-23T07:11:35.3094635Z Test assemblies : **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**
2019-09-23T07:11:35.3095999Z Test filter criteria : null
2019-09-23T07:11:35.3114780Z Search folder : C:\pool\_work8\s
2019-09-23T07:11:35.3125782Z Run settings file : C:\pool\_work8\s
2019-09-23T07:11:35.3144368Z Run in parallel : false
2019-09-23T07:11:35.3158394Z Run in isolation : false
2019-09-23T07:11:35.3243577Z Path to custom adapters : null
2019-09-23T07:11:35.3244967Z Other console options : null
2019-09-23T07:11:35.3269743Z Code coverage enabled : false
2019-09-23T07:11:35.3273300Z Diagnostics enabled : false
2019-09-23T07:11:35.3286973Z Rerun failed tests: false
2019-09-23T07:11:35.3308152Z VisualStudio version selected for test execution : latest
2019-09-23T07:11:35.7961563Z ========================================================
2019-09-23T07:11:50.3641983Z ======================================================
2019-09-23T07:12:16.2956956Z [command]"C:\Program Files (x86)\Microsoft Visual Studio17\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" @C:\pool\_work\_tempff2cb1-ddd1-11e9-a3ed-db0a48eb3ef5.txt
2019-09-23T07:12:16.9940929Z Microsoft (R) Test Execution Command Line Tool Version 15.9.0
2019-09-23T07:12:16.9941829Z Copyright (c) Microsoft Corporation. All rights reserved.
2019-09-23T07:12:16.9942101Z
2019-09-23T07:12:16.9942399Z vstest.console.exe
2019-09-23T07:12:16.9942679Z "C:\pool\_work8\s\REDACTED\bin\Release\REDACTED.Tests.dll"
2019-09-23T07:12:16.9942954Z /logger:"trx"
2019-09-23T07:12:16.9943238Z /TestAdapterPath:"C:\pool\_work8\s"
2019-09-23T07:12:17.6998070Z Starting test execution, please wait...
2019-09-23T07:12:20.5147395Z [xUnit.net 00:00:00.6296658] Discovering: REDACTED.Tests
2019-09-23T07:12:20.6880418Z [xUnit.net 00:00:00.8113825] Discovered: REDACTED.Tests
2019-09-23T07:12:20.6955571Z [xUnit.net 00:00:00.8192449] Starting: REDACTED.Tests
2019-09-23T07:12:21.9022641Z Passed REDACTED 1
2019-09-23T07:12:21.9024375Z Passed REDACTED 2
2019-09-23T07:12:21.9024811Z Passed REDACTED 3
2019-09-23T07:12:21.9025086Z Passed REDACTED 4
2019-09-23T07:12:21.9025460Z Passed REDACTED 5
2019-09-23T07:12:21.9026429Z Passed REDACTED 6
2019-09-23T07:12:21.9027211Z Passed REDACTED 7
2019-09-23T07:12:21.9027581Z Passed REDACTED 8
2019-09-23T07:12:21.9027938Z Passed REDACTED 9
2019-09-23T07:12:21.9028266Z [xUnit.net 00:00:01.9910390] Finished: REDACTED.Tests
2019-09-23T07:12:22.3918748Z Passed REDACTED 10
2019-09-23T07:12:22.3920336Z Passed REDACTED 11
2019-09-23T07:12:22.3920912Z Passed REDACTED 12
2019-09-23T07:12:22.3922185Z Passed REDACTED 13
2019-09-23T07:12:22.3923090Z Passed REDACTED 14
2019-09-23T07:12:22.3925098Z Passed REDACTED 15
2019-09-23T07:12:22.3926473Z Passed REDACTED 16
2019-09-23T07:12:22.3928334Z Passed REDACTED 17
2019-09-23T07:12:22.3930288Z Passed REDACTED 18
2019-09-23T07:12:22.4707999Z Results File: C:\pool\_work8\s\TestResults\REDACTED_2019-09-23_10_12_21.trx
2019-09-23T07:12:22.4743977Z
2019-09-23T07:12:22.4746692Z Total tests: 18. Passed: 18. Failed: 0. Skipped: 0.
2019-09-23T07:12:22.4748446Z Test Run Successful.
2019-09-23T07:12:22.4764305Z Test execution time: 2.7962 Seconds
2019-09-23T07:12:23.0702219Z ##[section]Async Command Start: Publish test results
2019-09-23T07:12:23.3336706Z Publishing test results to test run '432'
2019-09-23T07:12:23.3338983Z Test results remaining: 18. Test run id: 432
2019-09-23T07:12:24.1461381Z Published Test Run : http://REDACTED/Runs#runId=432&_a=runCharts
2019-09-23T07:12:24.1474239Z ##[section]Async Command End: Publish test results
2019-09-23T07:12:24.1476508Z ##[section]Finishing: run unit tests
根本原因是 Visual Studio 扫描测试的步骤有超过 100,000 个文件,在 Search folder
.
为了解决问题,我将搜索文件夹指向了文件夹树下方更具体的 bin 文件夹。这减少了扫描的文件数量,并显着缩短了整个步骤的 运行 时间。
在 DLL 中的 Azure DevOps Server build system installed on premises, I have a Build Pipeline with a Visual Studio Test step. The step is configured to discover and run XUnit 单元测试中。
步骤是通过 GUI 定义的。这是它的属性:
Version: 2.*
Select tests using: Test assemblies
Test files:
**\*test*.dll
!**\*TestAdapter.dll
!**\obj\**
Search folder: $(System.DefaultWorkingDirectory)
Select test platform using: Version
Test platform version: Latest
步骤 运行s 并成功通过,确定了 18 个测试。但是,这样做花费了不合理的时间:~51 秒。
运行 日志显示它在启动 vstest.console.exe
之前花费了大约 40 秒,似乎什么也没做。请在下面的完整日志中查看此行:
2019-09-23T07:11:35.7961563Z ========================================================
我的问题 - 为什么 Visual Studio 测试步骤会花费这么多时间空闲?我该怎么做才能减少它浪费的时间?
更新 1:作为实验,我创建了一个新的 git 存储库,其中仅包含我的解决方案构建的 (~100) 个 DLL。我创建了一个新管道,它只提取这些 DLL 和 运行s 单元测试。当我 运行 这个时,它 运行 快多了。特别是日志中原来需要 40 秒的两行现在只用了不到一秒。
更新 2:我试图通过指定测试 DLL 的完整路径并从 'Test files' 中删除所有 **
模式来加快速度 属性。这一步仍然 运行 和以前一样慢。
2019-09-23T07:11:32.4121152Z ##[section]Starting: run unit tests
2019-09-23T07:11:32.4135871Z ==============================================================================
2019-09-23T07:11:32.4136477Z Task : Visual Studio Test
2019-09-23T07:11:32.4136755Z Description : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2019-09-23T07:11:32.4137066Z Version : 2.143.8
2019-09-23T07:11:32.4137295Z Author : Microsoft Corporation
2019-09-23T07:11:32.4137526Z Help : [More information](https://go.microsoft.com/fwlink/?LinkId=835764)
2019-09-23T07:11:32.4138042Z ==============================================================================
2019-09-23T07:11:35.0894195Z SystemVssConnection exists true
2019-09-23T07:11:35.0937440Z SystemVssConnection exists true
2019-09-23T07:11:35.2406351Z Running tests using vstest.console.exe runner.
2019-09-23T07:11:35.2407337Z ======================================================
2019-09-23T07:11:35.2408408Z Test selector : Test assemblies
2019-09-23T07:11:35.2409125Z Test filter criteria : null
2019-09-23T07:11:35.2409694Z Search folder : C:\pool\_work8\s
2019-09-23T07:11:35.2412135Z VisualStudio version selected for test execution : latest
2019-09-23T07:11:35.2758339Z Run in parallel : false
2019-09-23T07:11:35.2810059Z Run in isolation : false
2019-09-23T07:11:35.2813902Z Path to custom adapters : null
2019-09-23T07:11:35.2831140Z Other console options : null
2019-09-23T07:11:35.2883526Z Code coverage enabled : false
2019-09-23T07:11:35.2884892Z Diagnostics enabled : false
2019-09-23T07:11:35.2937817Z SystemVssConnection exists true
2019-09-23T07:11:35.3039161Z Run the tests locally using vstest.console.exe
2019-09-23T07:11:35.3040102Z ========================================================
2019-09-23T07:11:35.3075806Z Test selector : Test assemblies
2019-09-23T07:11:35.3094635Z Test assemblies : **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**
2019-09-23T07:11:35.3095999Z Test filter criteria : null
2019-09-23T07:11:35.3114780Z Search folder : C:\pool\_work8\s
2019-09-23T07:11:35.3125782Z Run settings file : C:\pool\_work8\s
2019-09-23T07:11:35.3144368Z Run in parallel : false
2019-09-23T07:11:35.3158394Z Run in isolation : false
2019-09-23T07:11:35.3243577Z Path to custom adapters : null
2019-09-23T07:11:35.3244967Z Other console options : null
2019-09-23T07:11:35.3269743Z Code coverage enabled : false
2019-09-23T07:11:35.3273300Z Diagnostics enabled : false
2019-09-23T07:11:35.3286973Z Rerun failed tests: false
2019-09-23T07:11:35.3308152Z VisualStudio version selected for test execution : latest
2019-09-23T07:11:35.7961563Z ========================================================
2019-09-23T07:11:50.3641983Z ======================================================
2019-09-23T07:12:16.2956956Z [command]"C:\Program Files (x86)\Microsoft Visual Studio17\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" @C:\pool\_work\_tempff2cb1-ddd1-11e9-a3ed-db0a48eb3ef5.txt
2019-09-23T07:12:16.9940929Z Microsoft (R) Test Execution Command Line Tool Version 15.9.0
2019-09-23T07:12:16.9941829Z Copyright (c) Microsoft Corporation. All rights reserved.
2019-09-23T07:12:16.9942101Z
2019-09-23T07:12:16.9942399Z vstest.console.exe
2019-09-23T07:12:16.9942679Z "C:\pool\_work8\s\REDACTED\bin\Release\REDACTED.Tests.dll"
2019-09-23T07:12:16.9942954Z /logger:"trx"
2019-09-23T07:12:16.9943238Z /TestAdapterPath:"C:\pool\_work8\s"
2019-09-23T07:12:17.6998070Z Starting test execution, please wait...
2019-09-23T07:12:20.5147395Z [xUnit.net 00:00:00.6296658] Discovering: REDACTED.Tests
2019-09-23T07:12:20.6880418Z [xUnit.net 00:00:00.8113825] Discovered: REDACTED.Tests
2019-09-23T07:12:20.6955571Z [xUnit.net 00:00:00.8192449] Starting: REDACTED.Tests
2019-09-23T07:12:21.9022641Z Passed REDACTED 1
2019-09-23T07:12:21.9024375Z Passed REDACTED 2
2019-09-23T07:12:21.9024811Z Passed REDACTED 3
2019-09-23T07:12:21.9025086Z Passed REDACTED 4
2019-09-23T07:12:21.9025460Z Passed REDACTED 5
2019-09-23T07:12:21.9026429Z Passed REDACTED 6
2019-09-23T07:12:21.9027211Z Passed REDACTED 7
2019-09-23T07:12:21.9027581Z Passed REDACTED 8
2019-09-23T07:12:21.9027938Z Passed REDACTED 9
2019-09-23T07:12:21.9028266Z [xUnit.net 00:00:01.9910390] Finished: REDACTED.Tests
2019-09-23T07:12:22.3918748Z Passed REDACTED 10
2019-09-23T07:12:22.3920336Z Passed REDACTED 11
2019-09-23T07:12:22.3920912Z Passed REDACTED 12
2019-09-23T07:12:22.3922185Z Passed REDACTED 13
2019-09-23T07:12:22.3923090Z Passed REDACTED 14
2019-09-23T07:12:22.3925098Z Passed REDACTED 15
2019-09-23T07:12:22.3926473Z Passed REDACTED 16
2019-09-23T07:12:22.3928334Z Passed REDACTED 17
2019-09-23T07:12:22.3930288Z Passed REDACTED 18
2019-09-23T07:12:22.4707999Z Results File: C:\pool\_work8\s\TestResults\REDACTED_2019-09-23_10_12_21.trx
2019-09-23T07:12:22.4743977Z
2019-09-23T07:12:22.4746692Z Total tests: 18. Passed: 18. Failed: 0. Skipped: 0.
2019-09-23T07:12:22.4748446Z Test Run Successful.
2019-09-23T07:12:22.4764305Z Test execution time: 2.7962 Seconds
2019-09-23T07:12:23.0702219Z ##[section]Async Command Start: Publish test results
2019-09-23T07:12:23.3336706Z Publishing test results to test run '432'
2019-09-23T07:12:23.3338983Z Test results remaining: 18. Test run id: 432
2019-09-23T07:12:24.1461381Z Published Test Run : http://REDACTED/Runs#runId=432&_a=runCharts
2019-09-23T07:12:24.1474239Z ##[section]Async Command End: Publish test results
2019-09-23T07:12:24.1476508Z ##[section]Finishing: run unit tests
根本原因是 Visual Studio 扫描测试的步骤有超过 100,000 个文件,在 Search folder
.
为了解决问题,我将搜索文件夹指向了文件夹树下方更具体的 bin 文件夹。这减少了扫描的文件数量,并显着缩短了整个步骤的 运行 时间。