Fluentd+Kubernetes：根据标签隔离日志

Question

我想将 concat 过滤器应用于部署在 Kubernetes 上的 java 应用程序的日志，以将多行日志（不仅是异常）连接成一个日志事件..

这是修复问题后的最终版本。

想法是为部署添加标签

metadata:
  ...
spec:
  ...
  template:
    metadata:
      labels:
        logtype: springboot

Fluentd 配置：

# rewrite tag of events with kubernetes label kubernetes.labels.logtype=springboot
#
# it is important to change the tag. If the tag is not modified the event will be
# reemitted with the same tag and matched again by the rewrite tag filter -> infinite loop
<match kubernetes.var.log.containers.**>
  @type rewrite_tag_filter
  @log_level debug
  <rule>
    key $.kubernetes.labels.logtype
    pattern /^springboot$/
    tag springboot.${tag}
  </rule>
  # the rewrite tag filter is an event sink. Events that are not reemitted by the plugin
  # are gone. So we need a catch-all rule to reemitt any event that is not caught
  # by the spring boot rule.
  <rule>
    key log
    pattern /^.*$/
    # and the tag must be changed so that the event will skip the rewrite filter after reemitting
    tag unmatched.${tag}
  </rule>
</match>

# Handle multiline logs for springboot logs.
<filter springboot.**>
  @type concat
  key log
  separator ""
  multiline_start_regexp /^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d{3}\  (ERROR|WARN|INFO|DEBUG|TRACE)/
</filter>

Answer 1

<match **/> - 这要么有错字，要么是无效的 fluentd 配置
我需要查看完整的配置才能确定，但是 <match **> 也会匹配重写的标签，然后才会到达 <match springboot.**>。为避免这种情况，请将 match spring 引导放在 ** 匹配之前，或者将 ** 匹配缩小为来自 kube 的内容，例如<match kube.**>。 Re-tagged 事件被注入到管道的开始，并按照它们在配置中出现的顺序遍历它的各个部分。

Fluentd+Kubernetes：根据标签隔离日志

Fluentd+Kubernetes: Segregate logs based on label

fluentd

kubernetes

amazon-eks