Ansible wait_for 模块，从文件末尾开始

Question

使用 Ansible 中的 wait_for 模块，如果我在文件上使用 search_regex='foo'

它似乎从文件的开头开始，这意味着它将匹配旧数据，因此在重新启动 process/app (Java) 时它会附加到文件而不是启动新文件，wait_for 模块将退出 true 旧数据，但我想从文件的尾部检查。

Answer 1

wait_for 模块的 search_regex 中的正则表达式默认设置为多行。

您可以注册最后一行的内容，然后搜索该行之后出现的字符串（这里假设日志文件中没有重复行，即每行都包含一个时间戳）：

vars:
  log_file_to_check: <path_to_log_file>
  wanted_pattern: <pattern_to_match>

tasks:
  - name: Get the contents of the last line in {{ log_file_to_check }}
    shell: tail -n 1 {{ log_file_to_check }}
    register: tail_output

  - name: Create a variable with a meaningful name, just for clarity
    set_fact:
      last_line_of_the_log_file: "{{ tail_output.stdout }}"

  ### do some other tasks ###

  - name: Match "{{ wanted_pattern }}" appearing after "{{ last_line_of_the_log_file }}" in {{ log_file_to_check }}
    wait_for:
      path: "{{ log_file_to_check }}"
      search_regex: "{{ last_line_of_the_log_file }}\r(.*\r)*.*{{ wanted_pattern }}"

Answer 2

实际上，如果您可以在 java 应用程序日志文件上强制进行日志轮换，那么直接 wait_for 将实现您想要的，因为不会有任何历史日志行可以匹配

我正在使用这种方法滚动升级 mongodb 并在继续之前等待 mongod 日志中的 "waiting for connections"。

示例任务：

tasks:
  - name: Rotate mongod logs
    shell: kill -SIGUSR1 $(pidof mongod)
    args:
      executable: /bin/bash

  - name: Wait for mongod being ready
    wait_for:
      path: /var/log/mongodb/mongod.log
      search_regex: 'waiting for connections'

Answer 3

如果日志文件中的每一行都带有时间戳，

将起作用。否则，日志文件可能有多行与最后一行相同。

更 robust/durable 的方法是检查日志文件当前有多少行，然后搜索 'nth' 行之后出现的 regex/pattern。

vars:
  log_file: <path_to_log_file>
  pattern_to_match: <pattern_to_match>

tasks:
  - name: "Get contents of log file: {{ log_file }}"
    command: "cat {{ log_file }}"
    changed_when: false  # Do not show that state was "changed" since we are simply reading the file!
    register: cat_output

  - name: "Create variable to store line count (for clarity)"
    set_fact:
      line_count: "{{ cat_output.stdout_lines | length }}"

##### DO SOME OTHER TASKS (LIKE DEPLOYING APP) #####

  - name: "Wait until '{{ pattern_to_match}}' is found inside log file: {{ log_file }}"
    wait_for:
      path: "{{ log_file }}"
      search_regex: "^{{ pattern_to_skip_preexisting_lines }}{{ pattern_to_match }}$"
      state: present
    vars:
      pattern_to_skip_preexisting_lines : "(.*\n){% raw %}{{% endraw %}{{ line_count }},{% raw %}}{% endraw %}"  # i.e. if line_count=100, then this would equal "(.*\n){100,}"

Answer 4

另一种方法，使用中间临时文件来跟踪新记录：

- name: Create tempfile for log tailing
  tempfile:
    state: file
  register: tempfile
- name: Asynchronous tail log to temp file 
  shell: tail -n 0 -f /path/to/logfile > {{ tempfile.path }}
  async: 60
  poll: 0
- name: Wait for regex in log
  wait_for:
    path: "{{ tempfile.path }}"
    search_regex: 'some regex here'
- name: Remove tempfile
  file:
    path: "{{ tempfile.path }}"
    state: absent

Answer 5

我用上面@Slezhuk 的回答解决了我的问题。但是我发现了一个问题。 tail 命令不会在 playbook 完成后停止。它将永远运行。我不得不添加更多逻辑来停止该过程：

# the playbook doesn't stop the async job automatically. Need to stop it
- name: get the PID async tail
  shell: ps -ef | grep {{ tmpfile.path }} | grep tail | grep -v grep | awk '{print }'
  register: pid_grep
- set_fact:
    tail_pid: "{{pid_grep.stdout}}"
# the tail command shell has two processes. So need to kill both
- name: killing async tail command
  shell: ps -ef | grep " {{tail_pid}} " | grep -v grep | awk '{print }' | xargs kill -15
- name: Wait the tail process to be killed
  wait_for:
    path: /proc/{{tail_pid}}/status
    state: absent

Answer 6

在尝试做类似的事情时遇到了这个页面，自从上面的方法以来，ansible 已经出现了，所以这是我使用上面的解决方案。还使用了 timeout linux 命令和异步 - 所以加倍了一点。

如果应用程序在启动时轮换日志会容易得多 - 但有些应用程序不是那样好！

根据此页面.. https://docs.ansible.com/ansible/latest/user_guide/playbooks_async.html

The async tasks will run until they either complete, fail or timeout according to their async value.

希望对大家有所帮助

- name: set some common vars
  set_fact:
    tmp_tail_log_file: /some/path/tail_file.out # I did this as I didn't want to create a temp file every time, and i put it in the same spot.
    timeout_value: 300 # async will terminate the task - but to be double sure I used timeout linux command as well.
    log_file: /var/logs/my_log_file.log
    
  - name: "Asynchronous tail log to {{ tmp_tail_log_file }}"
    shell: timeout {{ timeout_value }} tail -n 0 -f {{ log_file }} > {{ tmp_tail_log_file }}
    async: "{{ timeout_value }}"
    poll: 0

  - name: "Wait for xxxxx to finish starting"
    wait_for:
      timeout: "{{ timeout_value }}"
      path: "{{ tmp_tail_log_file }}"
      search_regex: "{{ pattern_search }}"
    vars:
      pattern_search: (.*Startup process completed.*)
    register: waitfor

  - name: "Display log file entry"
    debug:
      msg: 
        - "xxxxxx startup completed - the following line was matched in the logs"
        - "{{ waitfor['match_groups'][0] }}"

Ansible wait_for 模块，从文件末尾开始

Ansible wait_for module, start at end of file

ansible

ansible-playbook