ELB 背后的 Salt masters 与 minions 有着脆弱的联系

Question

我正在运行AWS 进行以下设置：

两台 EC2 机器（Amazon Linux）前面的 Elastic Loadbalancer 带有一个 docker 容器，salt-master 运行位于
两个安装了 salt-minions 的 EC2 实例
minion config中的'master'值设置为负载均衡器的dns(SaltMaster-env-vpc-test.szfegmankg.us-east-1.elasticbeanstalk.com)
ELB 接受来自 minions 的所有流量
Salt-masters 接受来自 ELB 和 minions 的所有流量
Salt-masters PKI 文件夹在两个 master 之间共享
Salt-masters 拥有相同的私钥+public 密钥
制盐大师运行于 2017.7.1
2016.11.5 的 Salt-minions 运行（我用 2017.7.1 试过，结果一样）
Salt-minions 接受来自 ELB 和 masters 的所有流量

主配置如下所示：

open_mode: True 
worker_threads: 20
auto_accept: True 
log_level: error 
log_level_logfile: debug 
extension_modules: srv/salt/ext 
rest_cherrypy:   
port: 8000   
disable_ssl: True   
debug: True 
external_auth:   
  pam:
    saltdev:
      - .*
      - '@runner'
# Setting the job_cache to redis.
# The redis config settings are generated at the start of the docker container and
# will be written into /etc/salt/master.d/redis.conf 
master_job_cache: redis 
cache: redis 
pki_dir: /etc/salt/pki/master/efs

minion 配置如下：

id: WIN-AB3GO7BJ72I
log_file: C:\salt.log
multiprocessing: False
log_level_logfile: debug
pki_dir: /conf/pki/minion
master: SaltMaster-env-vpc-test.szfegmankg.us-east-1.elasticbeanstalk.com
master_type: str
master_alive_interval: 30
open_mode: True
root_dir: c:\salt
ipc_mode: tcp
recon_default: 1000
recon_max: 199000
recon_randomize: True

在 master 日志文件中，我可以在两个 master 上看到：

2017-09-05 10:06:18,118 [salt.utils.verify][DEBUG   ][35] This salt-master instance has accepted 2 minion keys.

两个主机上的 salt-key -L 产生相同的结果：

Accepted Keys:
WIN-AB3GO7BJ72I
WIN-EDMP9VB716B
Denied Keys:
Unaccepted Keys:
Rejected Keys:

看起来一切正常，一切正常。但是，test.ping 非常不稳定。有时它有效，但大多数时候无效。大多数时候，master 都没有从 minion 那里得到任何 return，而在 minion 端，我可以在日志中看到 minion 从来没有从 master 那里收到执行 'test.ping' 的消息。
示例 1：
test.ping 来自 Master1：

root@d7383ff8f8bf:/# salt 'WIN-EDMP9VB716B' test.ping
[ERROR   ] Exception raised when processing __virtual__ function for salt.loaded.int.cache.consul. Module will not be loaded: 'module' object has no attribute 'Consul'
[ERROR   ] An un-handled exception was caught by salt's global exception handler:
KeyError: 'redis.ls'
Traceback (most recent call last):
  File "/usr/bin/salt", line 10, in <module>
    salt_main()
  File "/usr/lib/python2.7/dist-packages/salt/scripts.py", line 476, in salt_main
    client.run()
  File "/usr/lib/python2.7/dist-packages/salt/cli/salt.py", line 173, in run
    for full_ret in cmd_func(**kwargs):
  File "/usr/lib/python2.7/dist-packages/salt/client/__init__.py", line 805, in cmd_cli
    **kwargs):
  File "/usr/lib/python2.7/dist-packages/salt/client/__init__.py", line 1597, in get_cli_event_returns
    connected_minions = salt.utils.minions.CkMinions(self.opts).connected_ids()
  File "/usr/lib/python2.7/dist-packages/salt/utils/minions.py", line 577, in connected_ids
    search = self.cache.ls('minions')
  File "/usr/lib/python2.7/dist-packages/salt/cache/__init__.py", line 244, in ls
    return self.modules[fun](bank, **self._kwargs)
  File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1113, in __getitem__
    func = super(LazyLoader, self).__getitem__(item)
  File "/usr/lib/python2.7/dist-packages/salt/utils/lazy.py", line 101, in __getitem__
    raise KeyError(key)
KeyError: 'redis.ls'
Traceback (most recent call last):
  File "/usr/bin/salt", line 10, in <module>
    salt_main()
  File "/usr/lib/python2.7/dist-packages/salt/scripts.py", line 476, in salt_main
    client.run()
  File "/usr/lib/python2.7/dist-packages/salt/cli/salt.py", line 173, in run
    for full_ret in cmd_func(**kwargs):
  File "/usr/lib/python2.7/dist-packages/salt/client/__init__.py", line 805, in cmd_cli
    **kwargs):
  File "/usr/lib/python2.7/dist-packages/salt/client/__init__.py", line 1597, in get_cli_event_returns
    connected_minions = salt.utils.minions.CkMinions(self.opts).connected_ids()
  File "/usr/lib/python2.7/dist-packages/salt/utils/minions.py", line 577, in connected_ids
    search = self.cache.ls('minions')
  File "/usr/lib/python2.7/dist-packages/salt/cache/__init__.py", line 244, in ls
    return self.modules[fun](bank, **self._kwargs)
  File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1113, in __getitem__
    func = super(LazyLoader, self).__getitem__(item)
  File "/usr/lib/python2.7/dist-packages/salt/utils/lazy.py", line 101, in __getitem__
    raise KeyError(key)
KeyError: 'redis.ls'

我知道redis的错误很快就会被修复https://github.com/saltstack/salt/issues/43295

示例 2：
test.ping 来自 Master1，在示例 1 后约 1 分钟：

root@d7383ff8f8bf:/# salt 'WIN-EDMP9VB716B' test.ping
WIN-EDMP9VB716B:
    True

同样在我的测试中，来自 Master2 的 test.ping 从未成功。

我想知道我的设置中是否存在一些我没有发现的缺陷，或者 Salt 是否仅适用于 HA 代理作为 ELB？

或者 Salt 在 ELB 后面根本不起作用？

Answer 1

有关更多答案，请参阅 https://github.com/saltstack/salt/issues/43368。

长话短说；博士因为 TCP 连接没有会话粘性，如果您使用 ELB 的 ip/name 作为入口点，目前无法使用 ELB 后面的 saltmaster。

ELB 背后的 Salt masters 与 minions 有着脆弱的联系

Salt masters behind ELB have flaky connection to minions

amazon-web-services

amazon-elb

salt-stack