根据 Logstash 中的事件消息添加字段不起作用

Question

我已经安装了 ELK 并在我的机器上工作，但现在我想根据事件消息进行更复杂的过滤和字段添加。

具体来说，我想根据消息模式设置"id_error"和"descripcio"。

我在 "logstash.conf" 文件中尝试了很多代码组合，但我无法获得预期的行为。

有人可以告诉我我做错了什么，我必须做什么，或者这不可能吗？提前致谢。

这是我的 "logstash.conf" 文件，我进行了最后一次测试，结果没有在 Kibana 中捕获任何事件：

input { 
    file {
        path => "C:\xxx.log"
    }
}

filter {
    grok {
        patterns_dir => "C:\elk\patterns"
        match => [ "message", "%{ERROR2:error2}" ]
        add_field => [ "id_error", "2" ]
        add_field => [ "descripcio", "error2!!!" ]
    }
    grok {
        patterns_dir => "C:\elk\patterns"
        match => [ "message", "%{ERROR1:error1}" ]
        add_field => [ "id_error", "1" ]
        add_field => [ "descripcio", "error1!!!" ]
    }
    if ("_grokparsefailure" in [tags]) { drop {} }
}

output {
  elasticsearch {
    host => "localhost"
    protocol => "http"
    index => "xxx-%{+YYYY.MM.dd}"
  }
}

我也试过下面的代码，导致字段 "id_error" 和 "descripcio" 都带有值“[1,2]”和“[error1!!!,error2!!!] " 分别在每个匹配的事件中。

由于 "break_on_match" 默认设置为 "true"，我希望只获取匹配子句后面的字段，但这并没有发生。

input { 
  file {
    path => "C:\xxx.log"
  }
}

filter {
  grok {
    patterns_dir => "C:\elk\patterns"
    match => [ "message", "%{ERROR1:error1}" ]
    add_field => [ "id_error", "1" ]
    add_field => [ "descripcio", "error1!!!" ]
    match => [ "message", "%{ERROR2:error2}" ]
    add_field => [ "id_error", "2" ]
    add_field => [ "descripcio", "error2!!!" ]
  }
  if ("_grokparsefailure" in [tags]) { drop {} }
}

output {
  elasticsearch {
    host => "localhost"
    protocol => "http"
    index => "xxx-%{+YYYY.MM.dd}"
  }
}

Answer 1

我已经解决了问题。我在 "logstash.conf":

中使用以下代码得到了预期的结果

input { 
  file {
    path => "C:\xxx.log"
  }
}

filter {
  grok {
    patterns_dir => "C:\elk\patterns"
    match => [ "message", "%{ERROR1:error1}" ]
    match => [ "message", "%{ERROR2:error2}" ]
  }
  if [message] =~ /error1_regex/ {
    grok {
        patterns_dir => "C:\elk\patterns"
        match => [ "message", "%{ERROR1:error1}" ]
    }
    mutate {
        add_field => [ "id_error", "1" ]
        add_field => [ "descripcio", "Error1!" ]
        remove_field => [ "message" ]
        remove_field => [ "error1" ]
    }
  }
  else if [message] =~ /error2_regex/ {
    grok {
        patterns_dir => "C:\elk\patterns"
        match => [ "message", "%{ERROR2:error2}" ]
    }
    mutate {
        add_field => [ "id_error", "2" ]
        add_field => [ "descripcio", "Error2!" ]
        remove_field => [ "message" ]
        remove_field => [ "error2" ]
    }
  }
  if ("_grokparsefailure" in [tags]) { drop {} }
}

output {
  elasticsearch {
    host => "localhost"
    protocol => "http"
    index => "xxx-%{+YYYY.MM.dd}"
  }
}

根据 Logstash 中的事件消息添加字段不起作用

Adding fields depending on event message in Logstash not working

elasticsearch

logstash

kibana

logstash-grok