LogStash-Grok：如何匹配Apache TimeStamp

Question

我不知道如何匹配我的时间戳。有人可以帮我吗？

Apache 日志中我的时间戳示例："2016-06-13T14:54:39.000+0100"

filter {   if [type] == "apache" {
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }   }
    date {
      timezone     => "UTC"
      match        => [ "timestamp" , "yyyy-mm-ddTHH:mm:ss Z"]
    }   } }

output {   stdout { codec => rubydebug } }

输出：

{:timestamp=>"2016-06-13T14:56:43.196000+0100", :message=>"Error: Cannot register filter date plugin. The error reported is: \n Illegal pattern component: T for pattern 'yyyy-mm-dd\THH:mm:ss Z'", :level=>:error}

Apache 日志示例：

{ "@version": "1", "@timestamp": "2016-06-14T09:11:23.000+0100", "message": "GET /page1/page2/ HTTP/1.1", "via": "192.168.1.1", "client-ip": "192.168.1.23", "remote-logname": "-", "remote-user": "-", "recv-time": "[14/Jun/2016:09:11:23 +0100]", "serve-time-microsec": "85471", "request": "GET /page1/page2/ HTTP/1.1", "status": "200", "size": "79648", "referer": "http://www.google.com/", "user-agent": "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13", "url": "/page1/page2/", "query": "", "method": "GET", "protocol": "HTTP/1.1", "vhost": "www.site.com", "received-size": "998" }

Answer 1

从你的完整日志中我可以看到它已经是 Json 格式，所以你不需要在使用 COMBINEDAPACHELOG 模式的消息字段上使用 grok 过滤器。只需使用以下格式的日期过滤器：

date {
     timezone  => "UTC"
     match     => [ "timestamp" , "yyyy-mm-dd'T'HH:mm:ss.SSSZ"]
}

这是我的完整 logstash.conf 文件，适用于您的示例：

input {
    file {
      path => "/var/log/test.log"
      codec => json
      start_position => "beginning"
    }
}
filter {
    date {
          timezone     => "UTC"
          match        => [ "timestamp" , "yyyy-mm-dd'T'HH:mm:ss.SSSZ"]
        }
}
output {
    stdout { 
        codec => rubydebug 
    }
}

LogStash-Grok：如何匹配Apache TimeStamp

LogStach-Grok: How to match Apache TimeStamp

apache

timestamp

grok

logstash

logstash-grok