在其中添加过滤器块时 Logstash 管道失败

Question

我正在创建 logstash 管道，我将日志文件作为输入并在 elasticsearch 上读取这些日志。我想在我的 logstash 管道配置中添加 geoip 过滤器，但是当我添加时它失败并正在关闭。

这是一个错误：

[2022-03-17T12:41:05,243][WARN ][logstash.outputs.elasticsearch][main] 
Elasticsearch Output configured with `ecs_compatibility => v8`, which 
resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common 
Schema. Once ECS v8 and an updated release of this plugin are publicly 
available, you will need to update this plugin to resolve this warning.
[2022-03-17T12:41:05,293][ERROR][logstash.javapipeline    ][main] 
Pipeline error {:pipeline_id=>"main", :exception=># 
<LogStash::ConfigurationError: GeoIP Filter in ECS-Compatiblity mode 
requires a `target` when `source` is not an `ip` sub-field, eg. [client] 
[ip]>, :backtrace=>["D:/logstash- 
8.1.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-7.2.11- 
java/lib/logstash/filters/geoip.rb:143:in `auto_target_from_source!'", 
"D:/logstash-8.1.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip- 
7.2.11-java/lib/logstash/filters/geoip.rb:133:in `setup_target_field'", 
"D:/logstash-8.1.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip- 
7.2.11-java/lib/logstash/filters/geoip.rb:108:in `register'", 
"org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in 
`register'", "D:/logstash-8.1.0/logstash- 
core/lib/logstash/java_pipeline.rb:232:in `block in register_plugins'", 
"org/jruby/RubyArray.java:1821:in `each'", "D:/logstash-8.1.0/logstash- 
core/lib/logstash/java_pipeline.rb:231:in `register_plugins'", 
"D:/logstash-8.1.0/logstash-core/lib/logstash/java_pipeline.rb:590:in 
`maybe_setup_out_plugins'", "D:/logstash-8.1.0/logstash- 
core/lib/logstash/java_pipeline.rb:244:in `start_workers'", 
"D:/logstash- 
8.1.0/logstash-core/lib/logstash/java_pipeline.rb:189:in `run'", 
"D:/logstash-8.1.0/logstash-core/lib/logstash/java_pipeline.rb:141:in `block in start'"], "pipeline.sources"=>["D:/logstash-8.1.0/my-logstash.conf"], :thread=>"#<Thread:0x6ea94258 run>"}
[2022-03-17T12:41:05,314][INFO ][logstash.javapipeline    ][main] Pipeline terminated {"pipeline.id"=>"main"}
[2022-03-17T12:41:05,357][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>8, :ecs_compatibility=>:v8}
[2022-03-17T12:41:05,390][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[2022-03-17T12:41:05,499][DEBUG][logstash.instrument.periodicpoller.os] Stopping
[2022-03-17T12:41:05,523][DEBUG][logstash.instrument.periodicpoller.jvm] Stopping
[2022-03-17T12:41:05,525][DEBUG][logstash.instrument.periodicpoller.persistentqueue] Stopping
[2022-03-17T12:41:05,532][DEBUG] 
[logstash.instrument.periodicpoller.deadletterqueue] Stopping
[2022-03-17T12:41:05,556][DEBUG][logstash.agent           ] Shutting 
down all pipelines {:pipelines_count=>0}

当我在没有过滤器的情况下使用以下配置时，它工作正常：

input {
 file {
  path => "D:/nest/es-logging-example/log/info/*.log"
  start_position => beginning
  sincedb_path => "NULL"
 }
}

output {
 elasticsearch {
    hosts => "localhost:9200"
    index => "myapplogs"
 }
 stdout{}
}

但是在配置文件中添加过滤器时失败并关闭:

input { 
 file {
  path => "D:/nest/es-logging-example/log/info/*.log"
  start_position => beginning
  sincedb_path => "NULL"
  }
}
filter {
 geoip {
    source => "clientip"
 }
}
output {
  elasticsearch {
    hosts => "localhost:9200"
    index => "myapplogs"
  }
  stdout{}
}

我在第二个配置中做错了什么？

Answer 1

报错是这样的

GeoIP Filter in ECS-Compatiblity mode requires a target when source is not an ip sub-field. You're simply missing an explicit target field

因此您的过滤器应如下所示：

filter {
 geoip {
    source => "clientip"
    target => "clientgeo"
 }
}

在其中添加过滤器块时 Logstash 管道失败

Logstash pipeline is failing when adding filter block in it

elasticsearch

logstash

kibana

elk