使用具有大型 YAML var 的循环时 Ansible 性能降低

Question

开发者社区您好！

我一直致力于开发一些 Ansible 剧本来管理 Citrix NetScaler 配置，并希望获得有关以下内容的一些帮助。我在名为 nsapp_lb_server:

的变量中定义了以下数据结构

nsapp_lb_server:
    - name:                      "SRV-1"
      ipaddress:                 "10.102.102.1"
      comment:                   "Chewbacca"

    - name:                      "SRV-2"
      ipaddress:                 "10.102.102.2"
      comment:                   "C-3PO"

    - name:                      "SRV-3"
      ipaddress:                 "10.102.102.3"
      comment:                   "Obi-Wan Kenobi"
...

[+ another 1200 item...]

我有以下任务：

  - name: "Check variables (loop)"
    ansible.builtin.assert:
        that:
            - ( (item.name is defined) and (item.name | length > 0) )
            - ( (item.ipaddress is defined) and (item.ipaddress | ipaddr() == item.ipaddress) )
            - ( (item.comment | length > 0) if (item.comment is defined) else omit )
    loop: "{{ nsapp_lb_server }}"

我的问题是，当我在 nsapp_lb_server 变量中有 数千条记录 时，循环非常慢 .任务在 30 分钟内完成，这是一个很长的时间...:-(

在网上查了一下，好像是Ansible的“loop”函数导致的问题，所以想看看有没有其他的方法可以使用而不是循环。

是否有任何 Ansible“循环”的替代方案可以提供相同的结果（遍历变量的条目）？我正在考虑使用 json_query，但仍然不知道如何在这种特定情况下实施它。

我的环境：

$ ansible --version
ansible [core 2.12.6]
  config file = /home/ansible/.ansible.cfg
  configured module search path = ['/home/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ansible/ansible/lib/python3.9/site-packages/ansible
  ansible collection location = /home/ansible/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/ansible/ansible/bin/ansible
  python version = 3.9.7 (default, Sep 21 2021, 00:13:39) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)]
  jinja version = 3.0.3
  libyaml = True

谁能给我指出正确的方向？我已经在我的代码集上工作了很长时间，在用大数据测试代码后，代码似乎没有用，因为运行时间。我还检查了分配给 Ansible 控制器运行的 VM 的硬件资源，没有问题。

非常感谢！

Answer 1

运行作为数千个单独任务的这种验证非常缓慢，因为它增加了大量执行和回调开销。您可以改为在单个任务中执行此操作，但需要注意的是，追踪无效列表项会更加困难：

- hosts: localhost
  gather_facts: false
  vars:
    nsapp_lb_server: "{{ nsapp_lb_samples * 10000 }}"
    nsapp_lb_samples:
        - name:                      "SRV-1"
          ipaddress:                 "10.102.102.1"
          comment:                   "Chewbacca"
        - name:                      "SRV-2"
          ipaddress:                 "10.102.102.2"
          comment:                   "C-3PO"
        - name:                      "SRV-3"
          ipaddress:                 "10.102.102.3"
          comment:                   "Obi-Wan Kenobi"
  tasks:
    - assert:
        that:
          - nsapp_lb_server | rejectattr('name') | length == 0
          - (nsapp_lb_server | map(attribute='ipaddress') | map('ipaddr')) == (nsapp_lb_server | map(attribute='ipaddress'))
          - nsapp_lb_server | selectattr('comment', 'defined') | rejectattr('comment') | length == 0

对于我输入的 30,000 个测试条目，它运行大约 5 秒。

为了更容易找到错误的值而不会使任务变得非常难看，您可以将其拆分为一系列任务：

- hosts: localhost
  gather_facts: false
  vars:
    nsapp_lb_server: "{{ nsapp_lb_samples * 10000 }}"
    nsapp_lb_samples:
        - name:                      "SRV-1"
          ipaddress:                 "10.102.102.1"
          comment:                   "Chewbacca"
        - name:                      "SRV-2"
          ipaddress:                 "10.102.102.2"
          comment:                   "C-3PO"
        - name:                      "SRV-3"
          ipaddress:                 "10.102.102.3"
          comment:                   "Obi-Wan Kenobi"
  tasks:
    - name: Check for missing names
      assert:
        that: nsapp_lb_server | rejectattr('name', 'defined') | length == 0
        fail_msg: "Bad entries: {{ nsapp_lb_server | rejectattr('name', 'defined') }}"

    - name: Check for bad names
      assert:
        that: nsapp_lb_server | rejectattr('name') | length == 0
        fail_msg: "Bad entries: {{ nsapp_lb_server | rejectattr('name') }}"

    - name: Check for missing IP addresses
      assert:
        that: nsapp_lb_server | rejectattr('ipaddress', 'defined') | length == 0
        fail_msg: "Bad entries: {{ nsapp_lb_server | rejectattr('ipaddress', 'defined') }}"

    - name: Check for bad IP addresses
      assert:
        that: (nsapp_lb_server | map(attribute='ipaddress') | map('ipaddr')) == (nsapp_lb_server | map(attribute='ipaddress'))
        fail_msg: "Suspicious values: {{ nsapp_lb_server | map(attribute='ipaddress') | map('ipaddr') | symmetric_difference(nsapp_lb_server | map(attribute='ipaddress')) }}"

    - name: Check for bad comments
      assert:
        that: nsapp_lb_server | selectattr('comment', 'defined') | rejectattr('comment') | length == 0
        fail_msg: "Bad entries: {{ nsapp_lb_server | selectattr('comment', 'defined') | rejectattr('comment') }}"

对于包含 30,000 个测试条目的相同列表，这将在大约 12 秒内运行。

使用具有大型 YAML var 的循环时 Ansible 性能降低

Slow Ansible performance when using loop with large YAML var

loops

ansible