如何使用logstash解析日志
How to parse logs using logstash
我有这样的消息需要 grok 过滤器解析:
"@timestamp":"2019-12-16T08:57:33.804Z","@version":"1","message":"[Optional[admin]]
(0.0.0.0, 0.0.0.0|0.0.0.0) 9999 approve
2019-12-16T08:57:30.414732Z","logger_name":"com.company.asd.asd.web.rest.MyClass","thread_name":"XNIO-1
task-5","level":"INFO","level_value":20000,"app_name":"asd","instance_id":"asd-123","app_port":"8080","version":"0.0.1-SNAPSHOT"
我尝试 http://grokdebug.herokuapp.com/ 解析我的日志,我写了这样的正则表达式来做到这一点:
"@timestamp":"%{TIMESTAMP_ISO8601:logTime}","@version":"%{INT:version}","message":"[\D*[%{WORD:login}]]
(%{IPV4:forwardedFor}\, %{IPV4:remoteAddr}\|%{IPV4:remoteAddr})
%{WORD:identificator} %{WORD:methodName}
%{TIMESTAMP_ISO8601:actionaDate}%{GREEDYDATA:all}
它似乎在这个调试器中工作,但是当我尝试将这一行添加到我在 .conf 文件中的过滤器时,它写入的所有内容都是 _grokparsefailure 并且我的消息保持不变,我的过滤器:
filter {
grok {
match => { "message" => ""@timestamp":"%{TIMESTAMP_ISO8601:logTime}","@version":"%{INT:version}","message":"\[\D*\[%{WORD:login}]\] \(%{IPV4:forwardedFor}\, %{IPV4:remoteAddr}\|%{IPV4:remoteAddr}\) %{WORD:identificator} %{WORD:methodName} %{TIMESTAMP_ISO8601:actionaDate}%{GREEDYDATA:all}" }
}
}
试试下面的 grok,
filter {
grok {
match => { "message" => "\"@timestamp\":\"%{TIMESTAMP_ISO8601:logTime}\",\"@version\":\"%{INT:version}\",\"message\":\"\[\D*\[%{WORD:login}]\] \(%{IPV4:forwardedFor}\, %{IPV4:remoteAddr}\|%{IPV4:remoteAddr}\) %{WORD:identificator} %{WORD:methodName} %{TIMESTAMP_ISO8601:actionaDate}%{GREEDYDATA:all}" }
}
}
我有这样的消息需要 grok 过滤器解析:
"@timestamp":"2019-12-16T08:57:33.804Z","@version":"1","message":"[Optional[admin]] (0.0.0.0, 0.0.0.0|0.0.0.0) 9999 approve 2019-12-16T08:57:30.414732Z","logger_name":"com.company.asd.asd.web.rest.MyClass","thread_name":"XNIO-1 task-5","level":"INFO","level_value":20000,"app_name":"asd","instance_id":"asd-123","app_port":"8080","version":"0.0.1-SNAPSHOT"
我尝试 http://grokdebug.herokuapp.com/ 解析我的日志,我写了这样的正则表达式来做到这一点:
"@timestamp":"%{TIMESTAMP_ISO8601:logTime}","@version":"%{INT:version}","message":"[\D*[%{WORD:login}]] (%{IPV4:forwardedFor}\, %{IPV4:remoteAddr}\|%{IPV4:remoteAddr}) %{WORD:identificator} %{WORD:methodName} %{TIMESTAMP_ISO8601:actionaDate}%{GREEDYDATA:all}
它似乎在这个调试器中工作,但是当我尝试将这一行添加到我在 .conf 文件中的过滤器时,它写入的所有内容都是 _grokparsefailure 并且我的消息保持不变,我的过滤器:
filter {
grok {
match => { "message" => ""@timestamp":"%{TIMESTAMP_ISO8601:logTime}","@version":"%{INT:version}","message":"\[\D*\[%{WORD:login}]\] \(%{IPV4:forwardedFor}\, %{IPV4:remoteAddr}\|%{IPV4:remoteAddr}\) %{WORD:identificator} %{WORD:methodName} %{TIMESTAMP_ISO8601:actionaDate}%{GREEDYDATA:all}" }
}
}
试试下面的 grok,
filter {
grok {
match => { "message" => "\"@timestamp\":\"%{TIMESTAMP_ISO8601:logTime}\",\"@version\":\"%{INT:version}\",\"message\":\"\[\D*\[%{WORD:login}]\] \(%{IPV4:forwardedFor}\, %{IPV4:remoteAddr}\|%{IPV4:remoteAddr}\) %{WORD:identificator} %{WORD:methodName} %{TIMESTAMP_ISO8601:actionaDate}%{GREEDYDATA:all}" }
}
}