在 LogStash 中写入@timestamp

Question

我需要将 UNIX 时间戳字段的值写入 @timestamp，以便我可以正确地索引流经 logstash 的数据，这部分工作正常。但是我也有要求@timestamp的值应该是插入时间。为此，我创建了一个临时字段来保存 @timestamp 的原始值。

这是我正在使用的内容：

filter {
    csv {
        separator => "  " # <- this white space is actually a tab, don't change it, it's already perfect
        skip_empty_columns => true
        columns => ["timestamp", ...]
    }
    # works just fine
    mutate {
        add_field => {
            "tmp" => "%{@timestamp}"
        }
    }
    # works just fine
    date {
       match => ["timestamp", "UNIX"]
       target => "@timestamp"
    }
    # this works too
    mutate {
        add_field => {
            "[@metadata][indexDate]" => "%{+YYYY-MM-dd}"
        }
    }   
    # @timestamp is not being set back to its original value
    date {
        match => ["tmp", "UNIX"]
        target => "@timestamp"
    }
    # works just fine
    mutate {
        remove_field => ["tmp"]
    }
}

output {
    elasticsearch {
        hosts => "elasticsearch:9200"
        # this works
        index => "indexname-%{[@metadata][indexDate]}"
    }
}

问题在这里：

date {
    match => ["tmp", "UNIX"]
    target => "@timestamp"
}

@timestamp 未被设置回其原始值。当我检查数据时，它与 timestamp 字段具有相同的值。

Answer 1

当您将日期添加到 tmp 时，它会以 ISO8601 格式添加，因此您需要使用：

date {
    match => ["tmp", "ISO8601"]
    target => "@timestamp"
}

在 LogStash 中写入@timestamp

Writing to @timestamp in LogStash

elasticsearch

logstash

logstash-file

logstash-configuration