json logstash 数组解析问题

Question

我们想使用JSON数组格式的logstash的http插件实现服务请求跟踪。

我们在尝试解析 JSON 数组时遇到以下错误：

错误：

:message=>"gsub mutation is only applicable for Strings, skipping", :field=>"message", :value=>nil, :level=>:debug, :file=>"logstash/filters/mutate.rb", :line=>"322", :method=>"gsub"}
:message=>"Exception in filterworker", "exception"=>#<LogStash::ConfigurationError: Only String and Array types are splittable. field:message is of type = NilClass>

我的 json 数组是：

{
    "data": [
        {
            "appName": "DemoApp",
            "appVersion": "1.1",
            "deviceId": "1234567",
            "deviceName": "moto e",
            "deviceOSVersion": "5.1",
            "packageName": "com.DemoApp",
            "message": "testing null pointer exception",
            "errorLog": "null pointer exception"
        },
        {
            "appName": "DemoApp",
            "appVersion": "1.1",
            "deviceId": "1234567",
            "deviceName": "moto e",
            "deviceOSVersion": "5.1",
            "packageName": "com.DemoApp",
            "message": "testing illegal state exception",
            "errorLog": "illegal state exception"
        }
    ]
}

我的 logstash 配置是：

    input {
            http {
            codec => "plain"
            }
    }
    filter{
            json {
                  source => "message"
                 }
            mutate { gsub => [ "message", "},", "shr" ] }
            split {
                  terminator => "shr"
                  field => "data"
           }
    }
    }


output {
 stdout { codec => "json" }
    gelf{
        host => localhost
        facility => "%{type}"
        level =>["%{SeverityLevel}", "INFO"]
        codec => "json"
    }
       file{
        path => "/chroot/result.log"
}
}

如有任何帮助，我们将不胜感激。

Answer 1

Logstash 有一个名为 message 的默认元数据字段。所以你的 json message 字段与它重叠。考虑将 json 字段名称 message 更改为另一个。

另一个选项可能使用 target 设置和引用目标字段，如：

json { source => "message" target => "data"}
mutate { gsub => [ "[data][message]", "\}\,\r\n\r\n\{", "\}shr\{" ] }

希望对您有所帮助。

json logstash 数组解析问题

json array parsing issue with logstash

json

logstash

graylog