Filebeat：监控两级目录层次结构

Question

我的问题

考虑以下目录结构：

/var/log/quodo/campaigns/deployment_29/campaign-32/users.log
/var/log/quodo/campaigns/deployment_29/campaign-12/ads.log
/var/log/quodo/campaigns/deployment_55/campaign-77/users.log
/var/log/quodo/campaigns/deployment_55/campaign-37/ads.log
...

我想使用 Filebeat 将 /var/log/quodo/campaigns/<whatever1>/<whatever2> 下的所有日志传送到 ElasticSearch。

Filebeat does not feature recursive monitoring of a directory:

To fetch all files from a predefined level of subdirectories, the following pattern can be used: /var/log/*/*.log.

This fetches all .log files from the subfolders of /var/log. It does not fetch log files from the /var/log folder itself. Currently it is not possible to recursively fetch all files in all subdirectories of a directory.

我尝试了什么

filebeat.prospectors:
- input_type: log
  paths:
    - /var/log/quodo/campaigns/*/*/*
  scan_frequency: 1s
output.elasticsearch:
   <connection data>

我的问题

我可以在 Filebeat 配置的目录层次结构中使用两层 * 吗？

Answer 1

以下是我的 filebeat 版本

$ filebeat.sh --version
filebeat version 5.6.2 (amd64), libbeat 5.6.2

我试过下面的配置

filebeat.prospectors:

- input_type: log

  paths:
    - /var/log/**/**/*


output.console:
  pretty: true

logging.level: debug

对我来说效果很好

{
  "@timestamp": "2017-10-07T18:12:17.694Z",
  "beat": {
    "hostname": "vagrant",
    "name": "vagrant",
    "version": "5.6.2"
  },
  "input_type": "log",
  "message": "tarun",
  "offset": 6,
  "source": "/var/log/test1/test3/test.log",
  "type": "log"
}

Answer 2

我在 filebeat 版本 7.6.1 上工作以监视递归

filebeat.prospectors:
  - input_type: log
  paths:
    - /var/log/**/*

Filebeat：监控两级目录层次结构

Filebeat: Monitor two levels of directory hierarchy

logging

filebeat

我的问题

我尝试了什么

我的问题