如何通过组合输入 csv 的两个字段/列在 logstash 中生成@timestamp

Question

我们在 csv 文件中有来自外部来源的数据，如下所示：

orderid,OrderDate,BusinessMinute,Quantity,Price
31874,01-01-2013,00:06,2,17.9

数据在一列中有 date，在另一列中有 time - 我需要通过将这两列组合在一起来生成时间戳。

我正在使用 csv filter 在 logstash 中使用以下配置从文件中读取上述数据 - 它正在生成自己的时间戳：

input {  
  file {
      path => "/root/data/import/Order.csv"
      start_position => "beginning"
  }
}
filter {
    csv {
        columns => ["orderid","OrderDate","BusinessMinute","Quantity","Price"]
        separator => ","
    } 
}
output {  
    elasticsearch {
        action => "index"
        host => "localhost"
        index => "demo"
        workers => 1
    }
}

如何将OrderDate + Business Minute组合成@timestamp？

Answer 1

使用 mutate filter to combine the OrderDate and BusinessMinute fields into a single (temporary) field, then use the date filter 并让它在成功时删除该字段。

filter {
  mutate {
    add_field => {
      "timestamp" => "%{OrderDate} %{BusinessMinute}"
    }
  }
  date {
    match => ["timestamp", "..."]
    remove_field => ["timestamp"]
  }
}

如何通过组合输入 csv 的两个字段/列在 logstash 中生成@timestamp

How to generate @timestamp in logstash by combining two fields / columns of input csv

csv

elasticsearch

logstash