ReadFrom.NEAREST 设置无法正常工作

Question

服务器和 Redis 设置

我在 AWS 上设置了 Redis 服务器，如下所示。

ap-northeast-1

10.0.100.21 redis/master @ap-northeast-1b
10.0.100.38 redis/replica#1 @ap-northeast-1c
10.0.100.62 redis/replica#2 @ap-northeast-1d

这些节点在同一个 VPC 中。 Redis 版本 = 6.0.9

[ec2-user@ip-10-0-100-21 redis-6.0.9]$ uname -a
Linux ip-10-0-100-21.ap-northeast-1.compute.internal 4.14.193-149.317.amzn2.x86_64 #1 SMP Thu Sep 3 19:04:44 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[ec2-user@ip-10-0-100-21 redis-6.0.9]$

Ping 响应时间

10.0.100.21

[ec2-user@ip-10-0-100-21 ~]$ ping -c 3 10.0.100.21
PING 10.0.100.21 (10.0.100.21) 56(84) bytes of data.
64 bytes from 10.0.100.21: icmp_seq=1 ttl=255 time=0.014 ms
64 bytes from 10.0.100.21: icmp_seq=2 ttl=255 time=0.039 ms
64 bytes from 10.0.100.21: icmp_seq=3 ttl=255 time=0.028 ms

--- 10.0.100.21 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2054ms
rtt min/avg/max/mdev = 0.014/0.027/0.039/0.010 ms
[ec2-user@ip-10-0-100-21 ~]$ ping -c 3 10.0.100.38
PING 10.0.100.38 (10.0.100.38) 56(84) bytes of data.
64 bytes from 10.0.100.38: icmp_seq=1 ttl=255 time=2.65 ms
64 bytes from 10.0.100.38: icmp_seq=2 ttl=255 time=2.60 ms
64 bytes from 10.0.100.38: icmp_seq=3 ttl=255 time=2.69 ms

--- 10.0.100.38 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 2.607/2.649/2.691/0.068 ms
[ec2-user@ip-10-0-100-21 ~]$ ping -c 3 10.0.100.62
PING 10.0.100.62 (10.0.100.62) 56(84) bytes of data.
64 bytes from 10.0.100.62: icmp_seq=1 ttl=255 time=1.97 ms
64 bytes from 10.0.100.62: icmp_seq=2 ttl=255 time=1.85 ms
64 bytes from 10.0.100.62: icmp_seq=3 ttl=255 time=2.01 ms

--- 10.0.100.62 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 1.857/1.950/2.014/0.067 ms
[ec2-user@ip-10-0-100-21 ~]$

10.0.100.38

[ec2-user@ip-10-0-100-38 ~]$ ping -c 3 10.0.100.21
PING 10.0.100.21 (10.0.100.21) 56(84) bytes of data.
64 bytes from 10.0.100.21: icmp_seq=1 ttl=255 time=2.62 ms
64 bytes from 10.0.100.21: icmp_seq=2 ttl=255 time=3.84 ms
64 bytes from 10.0.100.21: icmp_seq=3 ttl=255 time=2.63 ms

--- 10.0.100.21 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 2.621/3.032/3.846/0.577 ms
[ec2-user@ip-10-0-100-38 ~]$ ping -c 3 10.0.100.38
PING 10.0.100.38 (10.0.100.38) 56(84) bytes of data.
64 bytes from 10.0.100.38: icmp_seq=1 ttl=255 time=0.016 ms
64 bytes from 10.0.100.38: icmp_seq=2 ttl=255 time=0.027 ms
64 bytes from 10.0.100.38: icmp_seq=3 ttl=255 time=0.028 ms

--- 10.0.100.38 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2027ms
rtt min/avg/max/mdev = 0.016/0.023/0.028/0.007 ms
[ec2-user@ip-10-0-100-38 ~]$ ping -c 3 10.0.100.62
PING 10.0.100.62 (10.0.100.62) 56(84) bytes of data.
64 bytes from 10.0.100.62: icmp_seq=1 ttl=255 time=1.12 ms
64 bytes from 10.0.100.62: icmp_seq=2 ttl=255 time=1.05 ms
64 bytes from 10.0.100.62: icmp_seq=3 ttl=255 time=1.14 ms

--- 10.0.100.62 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 1.057/1.106/1.141/0.035 ms
[ec2-user@ip-10-0-100-38 ~]$

10.0.100.62

[ec2-user@ip-10-0-100-62 ~]$ ping -c 3 10.0.100.21
PING 10.0.100.21 (10.0.100.21) 56(84) bytes of data.
64 bytes from 10.0.100.21: icmp_seq=1 ttl=255 time=1.95 ms
64 bytes from 10.0.100.21: icmp_seq=2 ttl=255 time=2.01 ms
64 bytes from 10.0.100.21: icmp_seq=3 ttl=255 time=2.00 ms

--- 10.0.100.21 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 1.954/1.991/2.014/0.045 ms
[ec2-user@ip-10-0-100-62 ~]$ ping -c 3 10.0.100.38
PING 10.0.100.38 (10.0.100.38) 56(84) bytes of data.
64 bytes from 10.0.100.38: icmp_seq=1 ttl=255 time=1.08 ms
64 bytes from 10.0.100.38: icmp_seq=2 ttl=255 time=1.12 ms
64 bytes from 10.0.100.38: icmp_seq=3 ttl=255 time=1.26 ms

--- 10.0.100.38 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 1.082/1.155/1.260/0.085 ms
[ec2-user@ip-10-0-100-62 ~]$ ping -c 3 10.0.100.62
PING 10.0.100.62 (10.0.100.62) 56(84) bytes of data.
64 bytes from 10.0.100.62: icmp_seq=1 ttl=255 time=0.014 ms
64 bytes from 10.0.100.62: icmp_seq=2 ttl=255 time=0.029 ms
64 bytes from 10.0.100.62: icmp_seq=3 ttl=255 time=0.026 ms

--- 10.0.100.62 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2030ms
rtt min/avg/max/mdev = 0.014/0.023/0.029/0.006 ms
[ec2-user@ip-10-0-100-62 ~]$

根据以上，离每个节点最近的节点是：

距离 10.0.100.21 最近的节点是 10.0.100.21。
距离 10.0.100.38 最近的节点是 10.0.100.38。
距离 10.0.100.62 最近的节点是 10.0.100.62。

应用程序设置

Java

[ec2-user@ip-10-0-100-21 redis-6.0.9]$ java -version
openjdk version "11.0.9.1" 2020-11-04 LTS
OpenJDK Runtime Environment Corretto-11.0.9.12.1 (build 11.0.9.1+12-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.9.12.1 (build 11.0.9.1+12-LTS, mixed mode)
[ec2-user@ip-10-0-100-21 redis-6.0.9]$

Spring 引导 2.4.0
Spring 数据 Redis 2.4.1
生菜 6.0.1

RedisConfig.java

@Configuration
public class RedisConfig {
    @Value("${readFrom}")
    private String readFrom;

    @Bean
    public LettuceConnectionFactory redisConnectionFactory() {
        RedisStaticMasterReplicaConfiguration config = new RedisStaticMasterReplicaConfiguration("10.0.100.21", 6379);
        config.addNode("10.0.100.38", 6379);
        config.addNode("10.0.100.62", 6379);
        LettuceClientConfiguration clientConfig = LettuceClientConfiguration.builder()
                .readFrom(ReadFrom.valueOf(readFrom)).build();
        return new LettuceConnectionFactory(config, clientConfig);
    }
    
    @Bean
    public RedisTemplate<?,?> redisTemplate(RedisTemplate<?,?> template) {
        template.setConnectionFactory(redisConnectionFactory());
        StringRedisSerializer serializer = new StringRedisSerializer();
        JdkSerializationRedisSerializer jdkSerializer = new JdkSerializationRedisSerializer();
        
        template.setKeySerializer(serializer);
        template.setValueSerializer(jdkSerializer);
        template.setHashKeySerializer(serializer);
        template.setHashValueSerializer(jdkSerializer);
        return template;
    }
}

SampleRunner.java

@Component
public class SampleRunner implements ApplicationRunner {

    @Autowired
    private RedisTemplate<String, Object> redisTemplate;

    @Override
    public void run(ApplicationArguments args) throws Exception {
        System.out.println("Runner - start");
        for(int i=0; i<10; i++) {
            redisTemplate.opsForValue().get("key1");
            Thread.sleep(1000);
        }
        System.out.println("Runner - end");
    }

}

运行每个节点上的应用程序。

java -DreadFrom=nearest -jar redis-readfrom-sample-0.0.1-SNAPSHOT.jar

结果

运行 10.0.100.21 上的应用程序

[ec2-user@ip-10-0-100-62 redis-6.0.9]$ src/redis-cli monitor | grep GET
1605364234.819490 [0 10.0.100.21:57060] "GET" "key1"
1605364235.825171 [0 10.0.100.21:57060] "GET" "key1"
1605364236.829514 [0 10.0.100.21:57060] "GET" "key1"
1605364237.832915 [0 10.0.100.21:57060] "GET" "key1"
1605364238.836488 [0 10.0.100.21:57060] "GET" "key1"
1605364239.839737 [0 10.0.100.21:57060] "GET" "key1"
1605364240.843086 [0 10.0.100.21:57060] "GET" "key1"
1605364241.847348 [0 10.0.100.21:57060] "GET" "key1"
1605364242.850784 [0 10.0.100.21:57060] "GET" "key1"
1605364243.858162 [0 10.0.100.21:57060] "GET" "key1"

=> NG：获取命令应该发送到最近的节点（10.0.100.21），但发送到远节点（10.0.100.62）。

运行 10.0.100.38 上的应用程序

[ec2-user@ip-10-0-100-38 redis-6.0.9]$ src/redis-cli monitor | grep GET
1605364312.650791 [0 10.0.100.38:50216] "GET" "key1"
1605364313.655526 [0 10.0.100.38:50216] "GET" "key1"
1605364314.657089 [0 10.0.100.38:50216] "GET" "key1"
1605364315.659734 [0 10.0.100.38:50216] "GET" "key1"
1605364316.662639 [0 10.0.100.38:50216] "GET" "key1"
1605364317.664621 [0 10.0.100.38:50216] "GET" "key1"
1605364318.666050 [0 10.0.100.38:50216] "GET" "key1"
1605364319.668037 [0 10.0.100.38:50216] "GET" "key1"
1605364320.669549 [0 10.0.100.38:50216] "GET" "key1"
1605364321.671699 [0 10.0.100.38:50216] "GET" "key1"

=> 好的

运行 10.0.100.62 上的应用程序

[ec2-user@ip-10-0-100-62 redis-6.0.9]$ src/redis-cli monitor | grep GET
1605364353.037076 [0 10.0.100.62:48454] "GET" "key1"
1605364354.041437 [0 10.0.100.62:48454] "GET" "key1"
1605364355.044409 [0 10.0.100.62:48454] "GET" "key1"
1605364356.047743 [0 10.0.100.62:48454] "GET" "key1"
1605364357.050808 [0 10.0.100.62:48454] "GET" "key1"
1605364358.053005 [0 10.0.100.62:48454] "GET" "key1"
1605364359.054525 [0 10.0.100.62:48454] "GET" "key1"
1605364360.056585 [0 10.0.100.62:48454] "GET" "key1"
1605364361.058127 [0 10.0.100.62:48454] "GET" "key1"
1605364362.060059 [0 10.0.100.62:48454] "GET" "key1"

=> 好的

我需要任何额外的设置吗？

Answer 1

结论：ReadFrom.NEAREST仅在集群配置中有效。在 Sentinel 或主副本配置中无效。

ReadFrom.NEAREST 设置无法正常工作

ReadFrom.NEAREST settings doesn't work correctly

spring-data-redis

lettuce

服务器和 Redis 设置

Ping 响应时间

10.0.100.21

10.0.100.38

10.0.100.62

应用程序设置

Java

RedisConfig.java

SampleRunner.java

结果

运行 10.0.100.21 上的应用程序

运行 10.0.100.38 上的应用程序

运行 10.0.100.62 上的应用程序