使用测试服务器的 .NET 6 E2E 测试在 TeamCity CI 上失败，但手动工作 运行
.NET 6 E2E tests with test server fail on TeamCity CI, but work run manually
我有一个 .NET 6 应用程序（带有 ASP.NET 核心 Web 部件）。我使用 cypress 进行 E2E 测试。在我们从 .NET Framework 迁移到 .NET 6 之后，我想 运行 在 Web 应用程序上进行 E2E cypress 测试。
我的过程是，我实际上在 C# 中 运行 nUnit 测试初始化 in-memory SQLite 数据库，然后将此数据库连接用于我的应用程序。我使用这样的方法在实际测试运行之前创建端到端测试所需的数据。然后我从 C# 单元测试 运行 cypress。这一切在 .NET Framework 中运行良好，其中 Web 应用程序在测试期间实际上 运行，因此它在 IIS 中的真实 URL 上可用。
使用 .NET 6，我们有 WebApplicationFactory class 允许 运行 应用程序 in-memory。但是，我希望我的 Web 应用程序在具有真实端口的真实 URL 下可用，因此我可以针对它启动 cypress 测试。
在探索了 this GitHub discussion 的各种想法后，我创建了一个真实测试应用程序工厂（具有真实 IP 和端口）：
public abstract class RealTestServerWebAppFactory<TStartup> : WebApplicationFactory<TStartup> where TStartup : class
{
private bool _disposed;
private IHost _host;
public RealTestServerWebAppFactory()
{
ClientOptions.AllowAutoRedirect = false;
ClientOptions.BaseAddress = new Uri("http://localhost");
}
public string ServerAddress
{
get
{
EnsureServer();
return ClientOptions.BaseAddress.ToString();
}
}
public override IServiceProvider Services
{
get
{
EnsureServer();
return _host!.Services!;
}
}
// Code created with advices from https://github.com/dotnet/aspnetcore/issues/4892
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
base.ConfigureWebHost(builder);
// Final port will be dynamically assigned
builder.UseUrls("http://127.0.0.1:0");
builder.ConfigureServices(ConfigureServices);
}
protected override IHost CreateHost(IHostBuilder builder)
{
// Create the host for TestServer now before we
// modify the builder to use Kestrel instead.
var testHost = builder.Build();
// Modify the host builder to use Kestrel instead
// of TestServer so we can listen on a real address.
builder.ConfigureWebHost(webHostBuilder => webHostBuilder.UseKestrel());
// Create and start the Kestrel server before the test server,
// otherwise due to the way the deferred host builder works
// for minimal hosting, the server will not get "initialized
// enough" for the address it is listening on to be available.
// See https://github.com/dotnet/aspnetcore/issues/33846.
_host = builder.Build();
_host.Start();
// Extract the selected dynamic port out of the Kestrel server
// and assign it onto the client options for convenience so it
// "just works" as otherwise it'll be the default http://localhost
// URL, which won't route to the Kestrel-hosted HTTP server.
var server = _host.Services.GetRequiredService<IServer>();
var addresses = server.Features.Get<IServerAddressesFeature>();
ClientOptions.BaseAddress = addresses!.Addresses
.Select(x => new Uri(x))
.Last();
// Return the host that uses TestServer, rather than the real one.
// Otherwise the internals will complain about the host's server
// not being an instance of the concrete type TestServer.
// See https://github.com/dotnet/aspnetcore/pull/34702.
testHost.Start();
return testHost;
}
protected override void Dispose(bool disposing)
{
base.Dispose(disposing);
if (!_disposed)
{
if (disposing)
{
_host?.Dispose();
}
_disposed = true;
}
}
protected abstract void ConfigureServices(IServiceCollection services);
/// <summary>
/// Makes sure that the server has actually been run.
/// It must be executed at lest once to have the address and port assigned and the app fully working
/// </summary>
private void EnsureServer()
{
if (_host is null)
{
using var _ = CreateDefaultClient();
}
}
}
然后我创建了一个从 RealTestServerWebAppFactory 派生的 class，我在其中设置了一些测试依赖项注入，但这对我们的示例来说并不重要。我们称之为 child class MyServerTestApp.
测试中的实例创建如下所示：
var serverTestApp = new MyServerTestApp();
serverTestApp.ServerAddress;
在第二行，调用了 EnsureServer() 方法，它为应用程序创建了一个默认客户端。它还从 RealTestServerWebAppFactory class 调用 CreateHost(IHostBuilder builder) 来完成所有的魔法。在本地，在这一行之后我有应用程序的 URL，我可以通过网络浏览器（或 E2E 测试）访问它。
现在，问题是测试在本地运行良好。但是，我还在 CI - TeamCity 上执行它们，更准确地说。 TeamCity 用于 运行 那些测试的 dotnet test 命令如下所示：
"C:\Program Files\dotnet\dotnet.exe" test C:\TeamCity\buildAgent\work\c9a5fb5bca7f6666\Tests\E2ETests\bin\Release\net6.0-windows\MyApp.E2ETests.dll /logger:logger://teamcity /TestAdapterPath:C:\TeamCity\buildAgent\plugins\dotnet\tools\vstest15 /logger:console;verbosity=normal
这实际上是对 dotnet test 命令的直接使用。但是，TeamCity 执行这些测试失败，出现以下异常：
[12:47:54][dotnet test] A total of 1 test files matched the specified pattern.
[12:47:56][dotnet test] NUnit Adapter 4.2.0.0: Test execution started
[12:47:56][dotnet test] Running all tests in C:\TeamCity\buildAgent\work\c9a5fb5bca7f6666\Tests\E2ETests\bin\Release\net6.0-windows\MyApp.E2ETests.dll
[12:47:58][dotnet test] NUnit3TestExecutor discovered 10 of 10 NUnit test cases using Current Discovery mode, Non-Explicit run
[12:48:07][dotnet test] Setup failed for test fixture MyApp.E2ETests.SomeTestsClass
[12:48:07][dotnet test] System.InvalidOperationException : Sequence contains no elements
[12:48:07][dotnet test] StackTrace: at System.Linq.ThrowHelper.ThrowNoElementsException()
[12:48:07][dotnet test] at System.Linq.Enumerable.Last[TSource](IEnumerable`1 source)
[12:48:07][dotnet test] at MyApp.E2ETests.Infrastructure.RealTestServerWebAppFactory`1.CreateHost(IHostBuilder builder) in C:\TeamCity\buildAgent\work\c9a5fb5bca7f6666\Tests\MyApp.E2ETests\Infrastructure\RealTestServerWebAppFactory.cs:line 85
[12:48:07][dotnet test] at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.ConfigureHostBuilder(IHostBuilder hostBuilder)
[12:48:07][dotnet test] at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.EnsureServer()
[12:48:07][dotnet test] at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateDefaultClient(DelegatingHandler[] handlers)
[12:48:07][dotnet test] at MyApp.E2ETests.Infrastructure.RealTestServerWebAppFactory`1.EnsureServer() in C:\TeamCity\buildAgent\work\c9a5fb5bca7f6666\Tests\MyApp.E2ETests\Infrastructure\RealTestServerWebAppFactory.cs:line 122
[12:48:07][dotnet test] at MyApp.E2ETests.Infrastructure.RealTestServerWebAppFactory`1.get_ServerAddress() in C:\TeamCity\buildAgent\work\c9a5fb5bca7f6666\Tests\MyApp.E2ETests\Infrastructure\RealTestServerWebAppFactory.cs:line 35
System.InvalidOperationException : Sequence contains no elements
好像是这段代码：
var addresses = server.Features.Get<IServerAddressesFeature>();
returns TeamCity 运行 地址的空连接。因此，这段代码：
ClientOptions.BaseAddress = addresses!.Addresses
.Select(x => new Uri(x))
.Last();
失败 System.InvalidOperationException : Sequence contains no elements。
奇怪的是，如果我 运行 使用与 TeamCity 手动使用 cmd 相同的 dotnet test 命令进行这些测试，它就可以工作。它在同一台 CI 机器上运行，但从 cmd 手动执行，而不是作为 TeamCity 的构建步骤。 CI 机器 运行s Windows 服务器 2019.
我一直在想这可能与用于 运行ning TeamCity 的用户 rights/permissions 有关。但是，TeamCity 的 Windows 服务使用“本地系统”帐户，因此据我所知它应该具有所有权限。
感谢任何 help/ideas :)
根据 ，您必须使用以下代码在 IServerAddressesFeature 上设置默认地址：
var serverFeatures = featureCollection.Get<IServerAddressesFeature>();
if (serverFeatures.Addresses.Count == 0)
{
ListenOn(DefaultAddress); // Start the server on the default address
serverFeatures.Addresses.Add(DefaultAddress) // Add the default address to the IServerAddressesFeature
}
我有一个 .NET 6 应用程序（带有 ASP.NET 核心 Web 部件）。我使用 cypress 进行 E2E 测试。在我们从 .NET Framework 迁移到 .NET 6 之后，我想 运行 在 Web 应用程序上进行 E2E cypress 测试。
我的过程是，我实际上在 C# 中 运行 nUnit 测试初始化 in-memory SQLite 数据库，然后将此数据库连接用于我的应用程序。我使用这样的方法在实际测试运行之前创建端到端测试所需的数据。然后我从 C# 单元测试 运行 cypress。这一切在 .NET Framework 中运行良好，其中 Web 应用程序在测试期间实际上 运行，因此它在 IIS 中的真实 URL 上可用。
使用 .NET 6，我们有 WebApplicationFactory class 允许 运行 应用程序 in-memory。但是，我希望我的 Web 应用程序在具有真实端口的真实 URL 下可用，因此我可以针对它启动 cypress 测试。
在探索了 this GitHub discussion 的各种想法后，我创建了一个真实测试应用程序工厂（具有真实 IP 和端口）：
public abstract class RealTestServerWebAppFactory<TStartup> : WebApplicationFactory<TStartup> where TStartup : class
{
private bool _disposed;
private IHost _host;
public RealTestServerWebAppFactory()
{
ClientOptions.AllowAutoRedirect = false;
ClientOptions.BaseAddress = new Uri("http://localhost");
}
public string ServerAddress
{
get
{
EnsureServer();
return ClientOptions.BaseAddress.ToString();
}
}
public override IServiceProvider Services
{
get
{
EnsureServer();
return _host!.Services!;
}
}
// Code created with advices from https://github.com/dotnet/aspnetcore/issues/4892
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
base.ConfigureWebHost(builder);
// Final port will be dynamically assigned
builder.UseUrls("http://127.0.0.1:0");
builder.ConfigureServices(ConfigureServices);
}
protected override IHost CreateHost(IHostBuilder builder)
{
// Create the host for TestServer now before we
// modify the builder to use Kestrel instead.
var testHost = builder.Build();
// Modify the host builder to use Kestrel instead
// of TestServer so we can listen on a real address.
builder.ConfigureWebHost(webHostBuilder => webHostBuilder.UseKestrel());
// Create and start the Kestrel server before the test server,
// otherwise due to the way the deferred host builder works
// for minimal hosting, the server will not get "initialized
// enough" for the address it is listening on to be available.
// See https://github.com/dotnet/aspnetcore/issues/33846.
_host = builder.Build();
_host.Start();
// Extract the selected dynamic port out of the Kestrel server
// and assign it onto the client options for convenience so it
// "just works" as otherwise it'll be the default http://localhost
// URL, which won't route to the Kestrel-hosted HTTP server.
var server = _host.Services.GetRequiredService<IServer>();
var addresses = server.Features.Get<IServerAddressesFeature>();
ClientOptions.BaseAddress = addresses!.Addresses
.Select(x => new Uri(x))
.Last();
// Return the host that uses TestServer, rather than the real one.
// Otherwise the internals will complain about the host's server
// not being an instance of the concrete type TestServer.
// See https://github.com/dotnet/aspnetcore/pull/34702.
testHost.Start();
return testHost;
}
protected override void Dispose(bool disposing)
{
base.Dispose(disposing);
if (!_disposed)
{
if (disposing)
{
_host?.Dispose();
}
_disposed = true;
}
}
protected abstract void ConfigureServices(IServiceCollection services);
/// <summary>
/// Makes sure that the server has actually been run.
/// It must be executed at lest once to have the address and port assigned and the app fully working
/// </summary>
private void EnsureServer()
{
if (_host is null)
{
using var _ = CreateDefaultClient();
}
}
}
然后我创建了一个从 RealTestServerWebAppFactory 派生的 class，我在其中设置了一些测试依赖项注入，但这对我们的示例来说并不重要。我们称之为 child class MyServerTestApp.
测试中的实例创建如下所示：
var serverTestApp = new MyServerTestApp();
serverTestApp.ServerAddress;
在第二行，调用了 EnsureServer() 方法，它为应用程序创建了一个默认客户端。它还从 RealTestServerWebAppFactory class 调用 CreateHost(IHostBuilder builder) 来完成所有的魔法。在本地，在这一行之后我有应用程序的 URL，我可以通过网络浏览器（或 E2E 测试）访问它。
现在，问题是测试在本地运行良好。但是，我还在 CI - TeamCity 上执行它们，更准确地说。 TeamCity 用于 运行 那些测试的 dotnet test 命令如下所示：
"C:\Program Files\dotnet\dotnet.exe" test C:\TeamCity\buildAgent\work\c9a5fb5bca7f6666\Tests\E2ETests\bin\Release\net6.0-windows\MyApp.E2ETests.dll /logger:logger://teamcity /TestAdapterPath:C:\TeamCity\buildAgent\plugins\dotnet\tools\vstest15 /logger:console;verbosity=normal
这实际上是对 dotnet test 命令的直接使用。但是，TeamCity 执行这些测试失败，出现以下异常：
[12:47:54][dotnet test] A total of 1 test files matched the specified pattern.
[12:47:56][dotnet test] NUnit Adapter 4.2.0.0: Test execution started
[12:47:56][dotnet test] Running all tests in C:\TeamCity\buildAgent\work\c9a5fb5bca7f6666\Tests\E2ETests\bin\Release\net6.0-windows\MyApp.E2ETests.dll
[12:47:58][dotnet test] NUnit3TestExecutor discovered 10 of 10 NUnit test cases using Current Discovery mode, Non-Explicit run
[12:48:07][dotnet test] Setup failed for test fixture MyApp.E2ETests.SomeTestsClass
[12:48:07][dotnet test] System.InvalidOperationException : Sequence contains no elements
[12:48:07][dotnet test] StackTrace: at System.Linq.ThrowHelper.ThrowNoElementsException()
[12:48:07][dotnet test] at System.Linq.Enumerable.Last[TSource](IEnumerable`1 source)
[12:48:07][dotnet test] at MyApp.E2ETests.Infrastructure.RealTestServerWebAppFactory`1.CreateHost(IHostBuilder builder) in C:\TeamCity\buildAgent\work\c9a5fb5bca7f6666\Tests\MyApp.E2ETests\Infrastructure\RealTestServerWebAppFactory.cs:line 85
[12:48:07][dotnet test] at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.ConfigureHostBuilder(IHostBuilder hostBuilder)
[12:48:07][dotnet test] at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.EnsureServer()
[12:48:07][dotnet test] at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateDefaultClient(DelegatingHandler[] handlers)
[12:48:07][dotnet test] at MyApp.E2ETests.Infrastructure.RealTestServerWebAppFactory`1.EnsureServer() in C:\TeamCity\buildAgent\work\c9a5fb5bca7f6666\Tests\MyApp.E2ETests\Infrastructure\RealTestServerWebAppFactory.cs:line 122
[12:48:07][dotnet test] at MyApp.E2ETests.Infrastructure.RealTestServerWebAppFactory`1.get_ServerAddress() in C:\TeamCity\buildAgent\work\c9a5fb5bca7f6666\Tests\MyApp.E2ETests\Infrastructure\RealTestServerWebAppFactory.cs:line 35
System.InvalidOperationException : Sequence contains no elements
好像是这段代码：
var addresses = server.Features.Get<IServerAddressesFeature>();
returns TeamCity 运行 地址的空连接。因此，这段代码：
ClientOptions.BaseAddress = addresses!.Addresses
.Select(x => new Uri(x))
.Last();
失败 System.InvalidOperationException : Sequence contains no elements。
奇怪的是，如果我 运行 使用与 TeamCity 手动使用 cmd 相同的 dotnet test 命令进行这些测试，它就可以工作。它在同一台 CI 机器上运行，但从 cmd 手动执行，而不是作为 TeamCity 的构建步骤。 CI 机器 运行s Windows 服务器 2019.
我一直在想这可能与用于 运行ning TeamCity 的用户 rights/permissions 有关。但是，TeamCity 的 Windows 服务使用“本地系统”帐户，因此据我所知它应该具有所有权限。
感谢任何 help/ideas :)
根据 IServerAddressesFeature 上设置默认地址：
var serverFeatures = featureCollection.Get<IServerAddressesFeature>();
if (serverFeatures.Addresses.Count == 0)
{
ListenOn(DefaultAddress); // Start the server on the default address
serverFeatures.Addresses.Add(DefaultAddress) // Add the default address to the IServerAddressesFeature
}