无法获取 docker 个容器 运行 disque 以建立集群
Unable to get docker containers running disque to establish a cluster
我整理了一个docker集装箱建筑圆盘https://registry.hub.docker.com/u/jobflow/disque/
我能够部署并 运行 单个服务。我可以公开一个端口并从同一个容器或不同容器上的 disque 客户端 运行ning 连接到它。
但是当连接到一个实例并发送集群与另一个容器的 ip 和端口相遇时，它将尝试集群实例但最终失败。
我可以 link 两个容器，它们会很好地聚集在一起，但你不能 link 一个容器到多个容器。
我可以 运行 在单个容器中使用多个磁盘服务，它们可以正常集群。只是跨容器通信失败了。
例如：
3 disque servers running on 3 different containers
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0e31c5b751b5 jobflow/disque:latest "/bin/sh -c 'disque- 5 minutes ago Up 5 minutes 0.0.0.0:32770->7711/tcp disque-3
d48ec8e588d5 jobflow/disque:latest "/bin/sh -c 'disque- 5 minutes ago Up 5 minutes 0.0.0.0:32769->7711/tcp disque-2
8ee7ec27d210 jobflow/disque:latest "/bin/sh -c 'disque- 10 minutes ago Up 10 minutes 0.0.0.0:32768->7711/tcp disque
连接到服务器
# disque -h 192.168.99.100 -p 32768
192.168.99.100:32768>
加入集群
192.168.99.100:32768> cluster meet 192.168.99.100 32768
OK
192.168.99.100:32768>
看起来有效
192.168.99.100:32768> cluster info
cluster_state:ok
cluster_known_nodes:2
cluster_reachable_nodes:1
cluster_size:0
cluster_stats_messages_sent:171
cluster_stats_messages_received:0
192.168.99.100:32768>
等一下....不:(
192.168.99.100:32768> cluster info
cluster_state:ok
cluster_known_nodes:1
cluster_reachable_nodes:1
cluster_size:0
cluster_stats_messages_sent:296
cluster_stats_messages_received:0
192.168.99.100:32768>
让我们检查日志（将两个服务器都设置为调试）
192.168.99.100:32768> CONFIG SET loglevel debug
OK
7:P 17 Jun 21:11:25.357 * No cluster configuration found, I'm 3eb248db697774d0fa15e06ffcbf17f71767d4a0
Disque 0.0.1 (00000000/0) 64 bit
_ -
. Port: 7711
. o . PID: 7
.
- http://disque.io
7:P 17 Jun 21:11:25.398 # Server started, Disque version 0.0.1
7:P 17 Jun 21:11:25.399 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
7:P 17 Jun 21:11:25.399 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
7:P 17 Jun 21:11:25.399 * The server is now ready to accept connections on port 7711
7:P 17 Jun 22:42:20.608 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:42:25.695 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:42:30.790 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:42:35.901 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:42:40.988 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:42:46.069 - 1 clients connected, 724632 bytes in use
# disque -h 192.168.99.100 -p 32769
192.168.99.100:32769> CONFIG SET loglevel debug
OK
6:P 17 Jun 21:15:58.906 * No cluster configuration found, I'm cb52f64739b801286dbd76ceb8801ae38d43384e
Disque 0.0.1 (00000000/0) 64 bit
_ -
. Port: 7711
. o . PID: 6
.
- http://disque.io
6:P 17 Jun 21:15:58.920 # Server started, Disque version 0.0.1
6:P 17 Jun 21:15:58.920 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
6:P 17 Jun 21:15:58.921 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
6:P 17 Jun 21:15:58.921 * The server is now ready to accept connections on port 7711
6:P 17 Jun 22:44:52.162 - 1 clients connected, 724632 bytes in use
6:P 17 Jun 22:44:57.240 - 1 clients connected, 724632 bytes in use
6:P 17 Jun 22:45:02.318 - 1 clients connected, 724632 bytes in use
6:P 17 Jun 22:45:07.406 - 1 clients connected, 724632 bytes in use
6:P 17 Jun 22:45:12.482 - 1 clients connected, 724632 bytes in use
6:P 17 Jun 22:45:17.570 - 1 clients connected, 724632 bytes in use
我们再介绍一下集群，看看怎么样
# disque -h 192.168.99.100 -p 32768
192.168.99.100:32768> cluster meet 192.168.99.100 32769
OK
7:P 17 Jun 22:47:05.553 - 1 clients connected, 724600 bytes in use
7:P 17 Jun 22:47:09.838 . Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
7:P 17 Jun 22:47:09.838 . I/O error reading from node link: Connection refused
7:P 17 Jun 22:47:09.940 . Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
7:P 17 Jun 22:47:09.940 . I/O error reading from node link: Connection refused
7:P 17 Jun 22:47:10.041 . Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
7:P 17 Jun 22:47:10.041 . I/O error reading from node link: Connection refused
7:P 17 Jun 22:47:10.141 . Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
7:P 17 Jun 22:47:10.141 . I/O error reading from node link: Connection refused
7:P 17 Jun 22:47:10.244 . Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
7:P 17 Jun 22:47:24.673 . I/O error reading from node link: Connection refused
7:P 17 Jun 22:47:24.774 . Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
7:P 17 Jun 22:47:24.774 . I/O error reading from node link: Connection refused
7:P 17 Jun 22:47:25.895 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:47:30.979 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:47:36.083 - 1 clients connected, 724632 bytes in use
WAT?
Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
我通过了
192.168.99.100:32769
不是
192.168.99.100:42769
嗯嗯好的然后让我们通过
192.168.99.100:32768> cluster meet 192.168.99.100 22769
OK
7:P 17 Jun 22:52:08.996 . Connecting with Node 540d0f780c5bf04df56c89f25315743116f20e92 at 192.168.99.100:32769
这似乎行得通。但是...
192.168.99.100:32768> cluster info
cluster_state:ok
cluster_known_nodes:1
cluster_reachable_nodes:1
cluster_size:0
cluster_stats_messages_sent:445
cluster_stats_messages_received:0
问题应该是Docker使用了端口转发，与Disque目前的工作方式不兼容
但是您可以使用 1:1 映射在 Docker 中禁用端口转发，像这样：
$ docker run -d -p 7711:7711 ...
当这将在 Redis 集群中得到修复时，我也会将修复移植到 Disque 中。 修复 将使 Disque 实例能够报告 IP/port 对不同于节点将使用 getpeeraddr 系统调用通过自动检测感知到的一对。
我整理了一个docker集装箱建筑圆盘https://registry.hub.docker.com/u/jobflow/disque/
我能够部署并 运行 单个服务。我可以公开一个端口并从同一个容器或不同容器上的 disque 客户端 运行ning 连接到它。
但是当连接到一个实例并发送集群与另一个容器的 ip 和端口相遇时，它将尝试集群实例但最终失败。
我可以 link 两个容器，它们会很好地聚集在一起，但你不能 link 一个容器到多个容器。
我可以 运行 在单个容器中使用多个磁盘服务，它们可以正常集群。只是跨容器通信失败了。
例如：
3 disque servers running on 3 different containers
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0e31c5b751b5 jobflow/disque:latest "/bin/sh -c 'disque- 5 minutes ago Up 5 minutes 0.0.0.0:32770->7711/tcp disque-3
d48ec8e588d5 jobflow/disque:latest "/bin/sh -c 'disque- 5 minutes ago Up 5 minutes 0.0.0.0:32769->7711/tcp disque-2
8ee7ec27d210 jobflow/disque:latest "/bin/sh -c 'disque- 10 minutes ago Up 10 minutes 0.0.0.0:32768->7711/tcp disque
连接到服务器
# disque -h 192.168.99.100 -p 32768
192.168.99.100:32768>
加入集群
192.168.99.100:32768> cluster meet 192.168.99.100 32768
OK
192.168.99.100:32768>
看起来有效
192.168.99.100:32768> cluster info
cluster_state:ok
cluster_known_nodes:2
cluster_reachable_nodes:1
cluster_size:0
cluster_stats_messages_sent:171
cluster_stats_messages_received:0
192.168.99.100:32768>
等一下....不:(
192.168.99.100:32768> cluster info
cluster_state:ok
cluster_known_nodes:1
cluster_reachable_nodes:1
cluster_size:0
cluster_stats_messages_sent:296
cluster_stats_messages_received:0
192.168.99.100:32768>
让我们检查日志（将两个服务器都设置为调试）
192.168.99.100:32768> CONFIG SET loglevel debug
OK
7:P 17 Jun 21:11:25.357 * No cluster configuration found, I'm 3eb248db697774d0fa15e06ffcbf17f71767d4a0
Disque 0.0.1 (00000000/0) 64 bit
_ -
. Port: 7711
. o . PID: 7
.
- http://disque.io
7:P 17 Jun 21:11:25.398 # Server started, Disque version 0.0.1
7:P 17 Jun 21:11:25.399 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
7:P 17 Jun 21:11:25.399 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
7:P 17 Jun 21:11:25.399 * The server is now ready to accept connections on port 7711
7:P 17 Jun 22:42:20.608 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:42:25.695 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:42:30.790 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:42:35.901 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:42:40.988 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:42:46.069 - 1 clients connected, 724632 bytes in use
# disque -h 192.168.99.100 -p 32769
192.168.99.100:32769> CONFIG SET loglevel debug
OK
6:P 17 Jun 21:15:58.906 * No cluster configuration found, I'm cb52f64739b801286dbd76ceb8801ae38d43384e
Disque 0.0.1 (00000000/0) 64 bit
_ -
. Port: 7711
. o . PID: 6
.
- http://disque.io
6:P 17 Jun 21:15:58.920 # Server started, Disque version 0.0.1
6:P 17 Jun 21:15:58.920 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
6:P 17 Jun 21:15:58.921 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
6:P 17 Jun 21:15:58.921 * The server is now ready to accept connections on port 7711
6:P 17 Jun 22:44:52.162 - 1 clients connected, 724632 bytes in use
6:P 17 Jun 22:44:57.240 - 1 clients connected, 724632 bytes in use
6:P 17 Jun 22:45:02.318 - 1 clients connected, 724632 bytes in use
6:P 17 Jun 22:45:07.406 - 1 clients connected, 724632 bytes in use
6:P 17 Jun 22:45:12.482 - 1 clients connected, 724632 bytes in use
6:P 17 Jun 22:45:17.570 - 1 clients connected, 724632 bytes in use
我们再介绍一下集群，看看怎么样
# disque -h 192.168.99.100 -p 32768
192.168.99.100:32768> cluster meet 192.168.99.100 32769
OK
7:P 17 Jun 22:47:05.553 - 1 clients connected, 724600 bytes in use
7:P 17 Jun 22:47:09.838 . Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
7:P 17 Jun 22:47:09.838 . I/O error reading from node link: Connection refused
7:P 17 Jun 22:47:09.940 . Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
7:P 17 Jun 22:47:09.940 . I/O error reading from node link: Connection refused
7:P 17 Jun 22:47:10.041 . Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
7:P 17 Jun 22:47:10.041 . I/O error reading from node link: Connection refused
7:P 17 Jun 22:47:10.141 . Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
7:P 17 Jun 22:47:10.141 . I/O error reading from node link: Connection refused
7:P 17 Jun 22:47:10.244 . Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
7:P 17 Jun 22:47:24.673 . I/O error reading from node link: Connection refused
7:P 17 Jun 22:47:24.774 . Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
7:P 17 Jun 22:47:24.774 . I/O error reading from node link: Connection refused
7:P 17 Jun 22:47:25.895 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:47:30.979 - 1 clients connected, 724632 bytes in use
7:P 17 Jun 22:47:36.083 - 1 clients connected, 724632 bytes in use
WAT?
Connecting with Node 555abc8b9b37044e10e0a61fc28a3ce15b564696 at 192.168.99.100:42769
我通过了
192.168.99.100:32769
不是
192.168.99.100:42769
嗯嗯好的然后让我们通过
192.168.99.100:32768> cluster meet 192.168.99.100 22769
OK
7:P 17 Jun 22:52:08.996 . Connecting with Node 540d0f780c5bf04df56c89f25315743116f20e92 at 192.168.99.100:32769
这似乎行得通。但是...
192.168.99.100:32768> cluster info
cluster_state:ok
cluster_known_nodes:1
cluster_reachable_nodes:1
cluster_size:0
cluster_stats_messages_sent:445
cluster_stats_messages_received:0
问题应该是Docker使用了端口转发，与Disque目前的工作方式不兼容
但是您可以使用 1:1 映射在 Docker 中禁用端口转发，像这样：
$ docker run -d -p 7711:7711 ...
当这将在 Redis 集群中得到修复时，我也会将修复移植到 Disque 中。 修复 将使 Disque 实例能够报告 IP/port 对不同于节点将使用 getpeeraddr 系统调用通过自动检测感知到的一对。