Spring AMPQ 多个消费者与更高的预取值

Question

即使阅读了大量 SO 问题 (,) 和文章，仍不清楚为消费者设置哪个选项更好。多个消费者或更高的预取值？

据我了解，当谈到 SimpleRabbitListenerContainerFactory 时，~~因为它最初设计为每个连接只有一个线程~~，所以它旨在解决以下限制amqp-client 每个连接只有一个线程，这是否意味着设置多个消费者不会有太大区别，因为实际上只有一个线程从 rabbit 消费，而不是将它交给多个消费者（线程）？或者实际上有几个消费者同时消费？

那么，关于 spring 实施 rabbit 关于 prefetch/consumers 的最佳实践是什么？什么时候应该使用一个而不是另一个？我应该切换到这个新的 DirectRabbitListenerContainerFactory 吗？是 'better' 还是仅取决于用例？

当涉及到高预取时，我看到的一些缺点是，如果应用程序消耗了它可以保存在缓冲区中的更多消息，它可能会导致内存问题？（还没有实际测试过，tbh）

当涉及多个消费者时，我看到了在 OS 级别打开更多文件描述符的缺点，我看到 this 文章关于每个消费者实际上为每个 ack 和 ping 兔子这使它变慢。

仅供参考，如果相关的话，我通常会这样设置我的配置：

@Bean
public ConnectionFactory connectionFactory() {
final CachingConnectionFactory connectionFactory = new CachingConnectionFactory(server);
connectionFactory.setUsername(username);
connectionFactory.setPassword(password);
connectionFactory.setVirtualHost(virtualHost);
connectionFactory.setRequestedHeartBeat(requestedHeartBeat);
return connectionFactory;
}

@Bean
public AmqpAdmin amqpAdmin() {
    AmqpAdmin admin = new RabbitAdmin(connectionFactory());
    admin.declareQueue(getRabbitQueue());
    return admin;
}

@Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
final SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrentConsumers(concurrency);
factory.setMaxConcurrentConsumers(maxConcurrency);
factory.setPrefetchCount(prefetch);
factory.setMissingQueuesFatal(false);
return factory;
}

@Bean
public Queue getRabbitQueue() {
final Map<String, Object> p = new HashMap<String, Object>();
p.put("x-max-priority", 10);
return new Queue(queueName, true, false, false, p);
}

Answer 1

没有； SMLC 不是“为每个连接一个线程设计的”，它旨在解决 amqp-client 每个连接只有一个线程的限制，以便线程通过内存队列传递给消费者线程；这已不再是这种情况。客户端是多线程的，每个消费者有一个专用线程。

拥有多个消费者（增加并发性）是完全有效的（而且曾经是，即使是较旧的客户端）。

预取确实是为了减少网络干扰并提高整体吞吐量。您是否真的需要增加并发性与预取是正交的。如果 (a) 您的侦听器处理每条消息的速度相对较慢，并且 (b) 严格的消息排序并不重要，您通常会增加并发性。

引入 DirectListenerContainer 是为了提供不同的线程模型，其中直接在 amqp-client 线程上调用侦听器。

Choosing a Container 中描述了选择一个容器而不是另一个容器的原因。

The following features are available with the SMLC, but not the DMLC:

txSize - with the SMLC, you can set this to control how many messages are delivered in a transaction and/or to reduce the number of acks, but it may cause the number of duplicate deliveries to increase after a failure. (The DMLC does have mesagesPerAck which can be used to reduce the acks, the same as with txSize and the SMLC, but it can’t be used with transactions - each message is delivered and ack’d in a separate transaction).

maxConcurrentConsumers and consumer scaling intervals/triggers - there is no auto-scaling in the DMLC; it does, however, allow you to programmatically change the consumersPerQueue property and the consumers will be adjusted accordingly.

However, the DMLC has the following benefits over the SMLC:

Adding and removing queues at runtime is more efficient; with the SMLC, the entire consumer thread is restarted (all consumers canceled and re-created); with the DMLC, unaffected consumers are not canceled.

The context switch between the RabbitMQ Client thread and the consumer thread is avoided.

Threads are shared across consumers rather than having a dedicated thread for each consumer in the SMLC. However, see the IMPORTANT note about the connection factory configuration in the section called “Threading and Asynchronous Consumers”.

Spring AMPQ 多个消费者与更高的预取值

Spring AMPQ multiple consumers vs higher prefetch value

spring-boot

rabbitmq

spring-amqp