我的 lua 脚本将记录 ["key"] 返回为 nil

Question

我正在尝试 fliter/mask 使用 fluent bit 来更改日志 lua。该过滤器包含使用 gsub 将数字简单替换为 #。我正在以 JSON 格式输入日志 lua 无法访问键值。

test.lua

        function modifyRecord(tag, timestamp, record)
        local new_record = record
        old_message = record["message"]
        digits = "[%d]+"
        new_message = old_message:gsub(digits, "#")
        new_record["message"] = new_message
        return 1, timestamp, new_record
        end

日志文件 – test.log

 echo '{"Test": "Logs","message":"This is a message 123432"}' >> test.log

流利-bit.conf

    [SERVICE]
        # This is the main configuration block for fluent bit.
        # Ensure the follow line exists somewhere in the SERVICE block
        Plugins_File plugins.conf
    
    [INPUT]
        Name tail
        #Path /export/home/dummy/newrelic-infra/var/log/newrelic-infra/newrelic-infra.log
        Path /export/home/dummy/nrfb/test.log
   
    [OUTPUT]
        Name newrelic
        Match *
        licenseKey <key>
        proxy <proxy>
        endpoint <endpoint>
    
    [FILTER]
        Name modify
        Match *
        Add hostname <hostname>
        Add service_name newrelic-infra-fluent-bit
    
    [FILTER]
        Name    lua
        Match   *
        script  /export/home/dummy/nrfb/test.lua
        call    modifyRecord

Fluent 位日志

        -bash-4.2$ ./fluent-bit -c ./fluent-bit.conf
        Fluent Bit v1.6.3
        * Copyright (C) 2019-2020 The Fluent Bit Authors
        * Copyright (C) 2015-2018 Treasure Data
        * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
        * https://fluentbit.io
        
        [2022/04/15 11:09:20] [ info] [engine] started (pid=11454)
        [2022/04/15 11:09:20] [ info] [storage] version=1.0.6, initializing...
        [2022/04/15 11:09:20] [ info] [storage] in-memory
        [2022/04/15 11:09:20] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
        [2022/04/15 11:09:20] [ info] [sp] stream processor started
        [2022/04/15 11:09:20] [ info] [input:tail:tail.0] inotify_fs_add(): inode=194 watch_fd=1 name=/export/home/dummy/nrfb/test.log
        [2022/04/15 11:09:49] [error] [filter:lua:lua.1] error code 2: /export/home/dummy/nrfb/test.lua:5: attempt to index global 'old_message' (a nil value)

Answer 1

我的配置文件有误

[INPUT]
    Name tail
    #Path /export/home/dummy/newrelic-infra/var/log/newrelic-infra/newrelic-infra.log
    Path /export/home/dummy/nrfb/test.log

它应该将 json 的解析器设置为 look/parse。输入的更改和解析器的添加起到了作用。

[INPUT]
    Name tail
    #Path /export/home/dummy/newrelic-infra/var/log/newrelic-infra/newrelic-infra.log
    Path /export/home/dummy/nrfb/test.log
    Parser docker
[PARSER]
    Name        docker
    Format      json
    Time_Key    time
    Time_Format %Y-%m-%dT%H:%M:%S %z

我的 lua 脚本将记录 ["key"] 返回为 nil

My lua script is returning record["key"] as nil

lua

newrelic

fluent-bit