Logstash grok 链式条件过滤器

Question

我正在尝试为混合日志创建 grok 模式。这是我第一次创建条件链，我不断收到语法错误：

opt/logstash/bin/logstash -f /opt/logstash/conf.d/sip-parser.conf --    configtest
Error: Expected one of #, in, not , ==, !=, <=, >=, <, >, =~, !~, and, or,    xor, nand, { at line 27, column 14 (byte 580) after filter {
    # separate soap calls from responses
            grok {
                    match => { "message" => "\[%{TIMESTAMP_ISO8601:logdate}  \] %{LOGLEVEL:level}   %{GREEDYDATA:type}"}
                    }
                    if [type]

我的配置文件：

input {
    file{
            path => "/home/steven/sip.log"
            start_position => beginning
            # logstash stores the lastrun=> so we trick it
            sincedb_path => "/dev/null"

            #if logentry does not start with date it's part of previous entry
            codec =>  multiline {
            pattern => "\[^%{TIMESTAMP_ISO8601:logdate}\]"
            negate => "true"
            what => "previous"
    }

            }
    }

filter {
            grok {

                    match => { "message" => "\[%{TIMESTAMP_ISO8601:logdate}  \] %{LOGLEVEL:level}   %{GREEDYDATA:type}"}
                    }
                    # separate soap calls from responses
                    if ([type] ~= /AbstractLoggingInterceptor:\ Inbound Message$/) {
                            grok {
                                    match => { "message" => "\[%{TIMESTAMP_ISO8601:logdate}  \] %{LOGLEVEL:level}   %{GREEDYDATA:type}\n----------------------------\n%{GREEDYDATA:id}\n%{GREEDYDATA:responsecode}\n%{GREEDYDATA:encoding}\n%{GREEDYDATA:contenttype}\n%{GREEDYDATA:headers}\n%{GREEDYDATA:payload}\n--------------------------------------"}
                            }
                    }
                    else if ([type] ~= /AbstractLoggingInterceptor:\ Outbound Message$/) {
                            grok {
                                    match => {"message" => "\[%{TIMESTAMP_ISO8601:logdate}  \] %{LOGLEVEL:level}   %{GREEDYDATA:type}\n---------------------------\n%{GREEDYDATA:id}\n%{GREEDYDATA:responsecode}\n%{GREEDYDATA:encoding}\n%{GREEDYDATA:contenttype}\n%{GREEDYDATA:headers}\n%{GREEDYDATA:payload}\n--------------------------------------"}
                            }
                    }
                    else {
                            grok {
                                    match => {"message" => "\[%{TIMESTAMP_ISO8601:logdate}  \] %{LOGLEVEL:level}   %{GREEDYDATA:type}"}
                            }
                    }

            }

output {
    #elasticsearch {}
    stdout{}
}

我试图解析的日志文件可以在这里找到：http://pastebin.com/afbNfmjW 每个不同类型条目的单独 grok 模式已经在 http://grokdebug.herokuapp.com/ 中进行了测试，但我无法将它们链接在一起。我做错了什么？

Answer 1

您的条件 grok{} 不应位于第一个 grok 内，而应与其对等：

grok { ... }
if [myField] == "value" {
    grok { ... }
}

另请注意，您正在运行正则表达式以查看是否应该运行正则表达式。我建议将多个模式发送到一个 grok 节：

grok {
    match => { "myField",
        pattern1,
        pattern2,
        pattern3
    }
}

默认情况下，grok 将在一个匹配时停止处理它们。

Logstash grok 链式条件过滤器

Logstash grok chain conditional filters

regex

grok

logstash