SpecFlow BDD UI 在 Azure DevOps 管道上测试 运行
SpecFlow BDD UI tests running on Azure DevOps pipeline
我已经使用 SpecFlow BDD 和 Selenium 创建了一个测试套件,我正在尝试 运行 在 Azure DevOps 上使用管道:
trigger:
- master
pool:
name: Hosted Windows 2019 with VS2019
vmImage: windows-latest
schedules:
- cron: '00 16 * * MON,TUE,WED,THU,FRI'
displayName: Weekdays 4PM build
branches:
include:
- master
- releases/*
exclude:
- development
- feature/*
jobs:
- job: Build
displayName: Build
pool:
vmImage: windows-latest
steps:
- task: PowerShell@2
displayName: Clone the pipelines repository
inputs:
targetType: inline
script: >
git clone -b master
https://azuredevops:$($env:token)@dev.azure.com/abc-comapny/MyAzure/_git/MyProject.Builder
pipeline-repo/
env:
token: $(System.AccessToken)
- task: Bash@3
displayName: Prepare the nuget.config file
inputs:
targetType: inline
script: |
echo copying the nuget.config to the root folder
cp ./pipeline-repo/templates/nuget.config .
- task: NuGetCommand@2
displayName: Authenticate with Azure DevOps NuGet
inputs:
command: custom
arguments: >-
sources update -Name "Azure DevOps" -Username "vsts" -Password
"$(System.AccessToken)" -StorePasswordInClearText -ConfigFile
./nuget.config
- task: UseDotNet@2
displayName: Using SDK Version (3.1.x)
inputs:
version: 3.1.x
- task: DotNetCoreCLI@2
displayName: Restore
inputs:
command: restore
projects: '**/*.sln'
- task: DotNetCoreCLI@2
displayName: Build the solution
inputs:
command: build
projects: '**/*.sln'
- task: SpecFlowPlus@0
inputs:
projectFilePath: .
projectName: MyProject.Qa
projectLanguage: en
workItemPrefix: AUTO
- task: DotNetCoreCLI@2
displayName: Publish
inputs:
command: publish
publishWebProjects: false
projects: '**/*.sln'
arguments: >-
--configuration Release -o
$(build.artifactstagingdirectory)/SeleniumTests
zipAfterPublish: false
modifyOutputPath: false
- task: PublishBuildArtifacts@1
displayName: Publish Artifact
inputs:
PathtoPublish: $(build.artifactstagingdirectory)
condition: succeededOrFailed()
- script: >-
dotnet test MyProject.Qa.sln -r "$(Agent.TempDirectory)/" --logger
"trx;LogFilename=testresults.trx"
displayName: Run Unit Tests and Code Coverage
continueOnError: false
我已尝试查找以下位置给出的说明:
https://swimburger.net/blog/dotnet/how-to-run-net-core-selenium-ui-tests-on-azure-devops-pipelines
https://www.stuartwhiteford.com/running-selenium-ui-tests-in-an-azure-devops-pipeline/
步骤都通过了,但是我没有得到任何结果,也没有看到任何测试结果的输出。
我有 2 个项目,都是 MS 测试,一个用于 BDD 规范,另一个用于 Selenium 框架文件:
MyProject.Qa.Specification.csproj
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<IsPackable>false</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.7.1" />
<PackageReference Include="MSTest.TestAdapter" Version="2.1.2" />
<PackageReference Include="MSTest.TestFramework" Version="2.1.2" />
<PackageReference Include="coverlet.collector" Version="1.3.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Newtonsoft.Json" Version="12.0.3" />
<PackageReference Include="Selenium.Support" Version="3.141.0" />
<PackageReference Include="Selenium.WebDriver" Version="3.141.0" />
<PackageReference Include="Selenium.WebDriver.ChromeDriver" Version="85.0.4183.8700" />
<PackageReference Include="Selenium.WebDriver.GeckoDriver" Version="0.27.0" />
<PackageReference Include="SpecFlow" Version="3.4.3" />
<PackageReference Include="SpecFlow.Assist.Dynamic" Version="1.4.2" />
<PackageReference Include="SpecFlow.Tools.MsBuild.Generation" Version="3.4.3" />
<PackageReference Include="SpecRun.Runner" Version="3.4.12" />
<PackageReference Include="SpecRun.SpecFlow" Version="3.4.12" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\MyProject.Qa.Framework\MyProject.Qa.Framework.csproj" />
</ItemGroup>
<ItemGroup>
<Folder Include="FeatureFiles\" />
<Folder Include="Hooks\" />
<Folder Include="SpecFlowPlusRunner" />
<Folder Include="StepDefinitions\" />
</ItemGroup>
MyProject.Qa.Framework.csproj:
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<IsPackable>false</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="DotNetSeleniumExtras.PageObjects" Version="3.11.0" />
<PackageReference Include="DotNetSeleniumExtras.WaitHelpers" Version="3.11.0" />
<PackageReference Include="Newtonsoft.Json" Version="12.0.3" />
<PackageReference Include="NUnit" Version="3.12.0" />
<PackageReference Include="NUnit3TestAdapter" Version="3.17.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.7.1" />
<PackageReference Include="Selenium.Support" Version="3.141.0" />
<PackageReference Include="Selenium.WebDriver" Version="3.141.0" />
<PackageReference Include="Selenium.WebDriver.ChromeDriver" Version="85.0.4183.8700" />
<PackageReference Include="Selenium.WebDriver.GeckoDriver" Version="0.27.0" />
<PackageReference Include="Selenium.WebDriver.GeckoDriver.Win64" Version="0.27.0" />
<PackageReference Include="SpecFlow" Version="3.4.3" />
<PackageReference Include="System.Configuration.ConfigurationManager" Version="4.7.0" />
</ItemGroup>
<ItemGroup>
<Folder Include="CommonStructures\" />
<Folder Include="Controllers\" />
<Folder Include="Data\" />
<Folder Include="Dialogs\" />
<Folder Include="Extensions\" />
<Folder Include="Pages\" />
</ItemGroup>
有没有人使用 SpecFlow BDD UI 测试和 Azure DevOps Pipeline 来 运行 在非无头浏览器类型 运行 中进行 UI 测试?如果是这样,你能告诉我你做了什么吗?如果我对上面的 'azure-pipeline.yaml' 文件做错了什么并且需要对 运行 这些测试进行任何更改?
此外,我如何使用上述方法向所有利益相关者发送邮件?我可以使用 YAML 做到这一点,还是需要从 Azure DevOps 的其他部分做到这一点,一旦我有了报告?
我认为您缺少将测试结果发布到 Azure DevOps 的任务。您正在执行 dotnet test
并获得一个 trx
- 文件(结果文件),但您没有对它执行任何操作。
之后您需要使用 Publish Test Result
任务 (https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-test-results?view=azure-devops&tabs=yaml) 将 trx
- 文件上传到 Azure DevOps。
或者您更改 dotnet test
命令以使用 Visual Studio Test
任务 (https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/vstest?view=azure-devops)。此任务会自动上传 trx
- 文件。
我已经使用 SpecFlow BDD 和 Selenium 创建了一个测试套件,我正在尝试 运行 在 Azure DevOps 上使用管道:
trigger:
- master
pool:
name: Hosted Windows 2019 with VS2019
vmImage: windows-latest
schedules:
- cron: '00 16 * * MON,TUE,WED,THU,FRI'
displayName: Weekdays 4PM build
branches:
include:
- master
- releases/*
exclude:
- development
- feature/*
jobs:
- job: Build
displayName: Build
pool:
vmImage: windows-latest
steps:
- task: PowerShell@2
displayName: Clone the pipelines repository
inputs:
targetType: inline
script: >
git clone -b master
https://azuredevops:$($env:token)@dev.azure.com/abc-comapny/MyAzure/_git/MyProject.Builder
pipeline-repo/
env:
token: $(System.AccessToken)
- task: Bash@3
displayName: Prepare the nuget.config file
inputs:
targetType: inline
script: |
echo copying the nuget.config to the root folder
cp ./pipeline-repo/templates/nuget.config .
- task: NuGetCommand@2
displayName: Authenticate with Azure DevOps NuGet
inputs:
command: custom
arguments: >-
sources update -Name "Azure DevOps" -Username "vsts" -Password
"$(System.AccessToken)" -StorePasswordInClearText -ConfigFile
./nuget.config
- task: UseDotNet@2
displayName: Using SDK Version (3.1.x)
inputs:
version: 3.1.x
- task: DotNetCoreCLI@2
displayName: Restore
inputs:
command: restore
projects: '**/*.sln'
- task: DotNetCoreCLI@2
displayName: Build the solution
inputs:
command: build
projects: '**/*.sln'
- task: SpecFlowPlus@0
inputs:
projectFilePath: .
projectName: MyProject.Qa
projectLanguage: en
workItemPrefix: AUTO
- task: DotNetCoreCLI@2
displayName: Publish
inputs:
command: publish
publishWebProjects: false
projects: '**/*.sln'
arguments: >-
--configuration Release -o
$(build.artifactstagingdirectory)/SeleniumTests
zipAfterPublish: false
modifyOutputPath: false
- task: PublishBuildArtifacts@1
displayName: Publish Artifact
inputs:
PathtoPublish: $(build.artifactstagingdirectory)
condition: succeededOrFailed()
- script: >-
dotnet test MyProject.Qa.sln -r "$(Agent.TempDirectory)/" --logger
"trx;LogFilename=testresults.trx"
displayName: Run Unit Tests and Code Coverage
continueOnError: false
我已尝试查找以下位置给出的说明:
https://swimburger.net/blog/dotnet/how-to-run-net-core-selenium-ui-tests-on-azure-devops-pipelines
https://www.stuartwhiteford.com/running-selenium-ui-tests-in-an-azure-devops-pipeline/
步骤都通过了,但是我没有得到任何结果,也没有看到任何测试结果的输出。 我有 2 个项目,都是 MS 测试,一个用于 BDD 规范,另一个用于 Selenium 框架文件:
MyProject.Qa.Specification.csproj
<PropertyGroup> <TargetFramework>netcoreapp3.1</TargetFramework> <IsPackable>false</IsPackable> </PropertyGroup> <ItemGroup> <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.7.1" /> <PackageReference Include="MSTest.TestAdapter" Version="2.1.2" /> <PackageReference Include="MSTest.TestFramework" Version="2.1.2" /> <PackageReference Include="coverlet.collector" Version="1.3.0"> <PrivateAssets>all</PrivateAssets> <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets> </PackageReference> <PackageReference Include="Newtonsoft.Json" Version="12.0.3" /> <PackageReference Include="Selenium.Support" Version="3.141.0" /> <PackageReference Include="Selenium.WebDriver" Version="3.141.0" /> <PackageReference Include="Selenium.WebDriver.ChromeDriver" Version="85.0.4183.8700" /> <PackageReference Include="Selenium.WebDriver.GeckoDriver" Version="0.27.0" /> <PackageReference Include="SpecFlow" Version="3.4.3" /> <PackageReference Include="SpecFlow.Assist.Dynamic" Version="1.4.2" /> <PackageReference Include="SpecFlow.Tools.MsBuild.Generation" Version="3.4.3" /> <PackageReference Include="SpecRun.Runner" Version="3.4.12" /> <PackageReference Include="SpecRun.SpecFlow" Version="3.4.12" /> </ItemGroup> <ItemGroup> <ProjectReference Include="..\MyProject.Qa.Framework\MyProject.Qa.Framework.csproj" /> </ItemGroup> <ItemGroup> <Folder Include="FeatureFiles\" /> <Folder Include="Hooks\" /> <Folder Include="SpecFlowPlusRunner" /> <Folder Include="StepDefinitions\" /> </ItemGroup>
MyProject.Qa.Framework.csproj:
<PropertyGroup> <TargetFramework>netcoreapp3.1</TargetFramework> <IsPackable>false</IsPackable> </PropertyGroup> <ItemGroup> <PackageReference Include="DotNetSeleniumExtras.PageObjects" Version="3.11.0" /> <PackageReference Include="DotNetSeleniumExtras.WaitHelpers" Version="3.11.0" /> <PackageReference Include="Newtonsoft.Json" Version="12.0.3" /> <PackageReference Include="NUnit" Version="3.12.0" /> <PackageReference Include="NUnit3TestAdapter" Version="3.17.0"> <PrivateAssets>all</PrivateAssets> <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets> </PackageReference> <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.7.1" /> <PackageReference Include="Selenium.Support" Version="3.141.0" /> <PackageReference Include="Selenium.WebDriver" Version="3.141.0" /> <PackageReference Include="Selenium.WebDriver.ChromeDriver" Version="85.0.4183.8700" /> <PackageReference Include="Selenium.WebDriver.GeckoDriver" Version="0.27.0" /> <PackageReference Include="Selenium.WebDriver.GeckoDriver.Win64" Version="0.27.0" /> <PackageReference Include="SpecFlow" Version="3.4.3" /> <PackageReference Include="System.Configuration.ConfigurationManager" Version="4.7.0" /> </ItemGroup> <ItemGroup> <Folder Include="CommonStructures\" /> <Folder Include="Controllers\" /> <Folder Include="Data\" /> <Folder Include="Dialogs\" /> <Folder Include="Extensions\" /> <Folder Include="Pages\" /> </ItemGroup>
有没有人使用 SpecFlow BDD UI 测试和 Azure DevOps Pipeline 来 运行 在非无头浏览器类型 运行 中进行 UI 测试?如果是这样,你能告诉我你做了什么吗?如果我对上面的 'azure-pipeline.yaml' 文件做错了什么并且需要对 运行 这些测试进行任何更改?
此外,我如何使用上述方法向所有利益相关者发送邮件?我可以使用 YAML 做到这一点,还是需要从 Azure DevOps 的其他部分做到这一点,一旦我有了报告?
我认为您缺少将测试结果发布到 Azure DevOps 的任务。您正在执行 dotnet test
并获得一个 trx
- 文件(结果文件),但您没有对它执行任何操作。
之后您需要使用 Publish Test Result
任务 (https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-test-results?view=azure-devops&tabs=yaml) 将 trx
- 文件上传到 Azure DevOps。
或者您更改 dotnet test
命令以使用 Visual Studio Test
任务 (https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/vstest?view=azure-devops)。此任务会自动上传 trx
- 文件。