Spring Webflux 性能测试抛出 PoolAcquirePendingLimitException

Question

我正在努力学习 Spring webflux。我编写了以下代码来测试响应式编程的性能。这是我的一项服务的控制器：

@RestController
public class PlayerController {

    @Autowired
    private PlayerRepository playerRepository;

    @GetMapping("/players/{id}")
    public Mono<Player> getPlayerById(@PathVariable int id) {
        return playerRepository.findById(id);
    }

型号class:

@Data
@NoArgsConstructor
@AllArgsConstructor
public class Player {

    @Id
    private Integer id;
    private String name;
    private Integer age;
}

存储库：

@Repository
public interface PlayerRepository extends ReactiveCrudRepository<Player, Integer> {}

以下是调用上述服务的客户端，然后执行数据库调用，然后检查它们是否返回相同的数据。客户端的存储库和模型相同。

@RestController
public class PlayerController {

    @Autowired
    private PlayerRepository playerRepository;

    @Autowired
    private WebClient webClient;

    @GetMapping("/players/{id}")
    public Mono<Response> getPlayerById(@PathVariable int id) {
        LocalDateTime start = LocalDateTime.now();
        Mono<Player> playerExternal = webClient.get().uri("/players/{id}", id).retrieve().bodyToMono(Player.class);
        Mono<Player> playerDB = playerRepository.findById(2);


        Mono<Response> result = playerExternal.zipWith(playerDB, (ext, db) -> {
            Response response = new Response();
            response.setFlag(ext.getId() == db.getId());
            LocalDateTime end = LocalDateTime.now();
            response.setTimeTaken(end.from(start).until(end, ChronoUnit.MILLIS) + " ms");

            return response;

        });

        return result;
    }

pom.xml

    <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.5.0</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.example</groupId>
    <artifactId>reactive-spring</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>reactive-spring</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-webflux</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.projectreactor</groupId>
            <artifactId>reactor-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-r2dbc</artifactId>
        </dependency>
        <dependency>
            <groupId>dev.miku</groupId>
            <artifactId>r2dbc-mysql</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

客户端在8080端口运行，调用服务在9090端口运行。我使用的是spring boot 2.5.0。我也在使用 jmeter 来模拟 10000 个并发请求。在执行 jmeter 测试计划时，出现以下异常，

org.springframework.web.reactive.function.client.WebClientRequestException: Pending acquire queue has reached its maximum size of 1000; nested exception is reactor.netty.internal.shaded.reactor.pool.PoolAcquirePendingLimitException: Pending acquire queue has reached its maximum size of 1000
    at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException(ExchangeFunctions.java:141) ~[spring-webflux-5.3.7.jar:5.3.7]
    Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
    |_ checkpoint ⇢ Request to GET http://localhost:9090/players/1 [DefaultWebClient]
    |_ checkpoint ⇢ Handler com.example.demo.controller.PlayerController#getPlayerById(int) [DispatcherHandler]
    |_ checkpoint ⇢ HTTP GET "/players/1" [ExceptionHandlingWebHandler]
Stack trace:
        at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException(ExchangeFunctions.java:141) ~[spring-webflux-5.3.7.jar:5.3.7]
        at reactor.core.publisher.MonoErrorSupplied.subscribe(MonoErrorSupplied.java:70) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.Mono.subscribe(Mono.java:4150) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.MonoNext$NextSubscriber.onError(MonoNext.java:93) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onError(MonoFlatMapMany.java:204) ~[reactor-core-3.4.6.jar:3.4.6]

如果我需要提供完整的堆栈跟踪，请告诉我。如果我正在做任何我不应该做的事情，你能告诉我吗？我也可以分享 jmeter 测试结果。我看到很多请求都失败了，其中大部分都需要很长时间（范围在 1 秒到 18 秒之间）才能响应。

Answer 1

默认情况下WebClient 使用连接池运行。池的默认设置是最大 500 个连接和最大 1000 个挂起请求。您有 JMeter 并尝试模拟 10000，但您没有指定如何分配负载。您可能需要增加最大待处理请求数。看看这个 documentation and this documentation.

如果你想配置 WebClient 那么你需要：

@Bean
public ReactorResourceFactory resourceFactory() {
    ConnectionProvider provider =
            ConnectionProvider.builder("test")
                              .maxConnections(500)
                              // Set custom max pending requests
                              .pendingAcquireMaxCount(...)
                              .build();
    ReactorResourceFactory factory = new ReactorResourceFactory();
    factory.setUseGlobalResources(false); 
    factory.setConnectionProvider(provider);
    return factory;
}

@Bean
public WebClient webClient() {

    Function<HttpClient, HttpClient> mapper = client -> {
        // Further customizations...
    };

    ClientHttpConnector connector =
            new ReactorClientHttpConnector(resourceFactory(), mapper); 

    return WebClient.builder().clientConnector(connector).build(); 
}

Spring Webflux 性能测试抛出 PoolAcquirePendingLimitException

Spring Webflux Performance Test throwing PoolAcquirePendingLimitException

spring

spring-boot

reactor-netty

spring-webflux

spring-webclient