Logstash 管道中止

Question

我一直收到这个错误。我正在尝试解析 csv。文件。我想知道我是否缺少图书馆或其他东西。

我在 windows 命令行中使用 logstash.bat -f logstash.conf 命令来运行并获得此输出。

我正在尝试使用 rubydebug 编解码器进行输出

21:19:03.781 [main] INFO  logstash.setting.writabledirectory - Creating directory {:setting=>"path.queue", :path=>"C:/Users/Public/logstash-5.2.1/data/queue"}

21:19:03.787 [LogStash::Runner] INFO  logstash.agent - No persistent UUID file found. Generating new UUID {:uuid=>"0546332b-dc4d-4916-b5c6-7900d1fdd8a4", :path=>"C:/Users/Public/logstash-5.2.1/data/uuid"}

21:19:04.138 [[main]-pipeline-manager] ERROR logstash.agent - Pipeline aborted due to error {:exception=>#<LogStash::ConfigurationError: translation missing: en.logstash.agent.configuration.invalid_plugin_register>, :backtrace=>["C:/Users/Public/logstash-5.2.1/vendor/bundle/jruby/1.9/gems/logstash-filter-mutate-3.1.3/lib/logstash/filters/mutate.rb:178:in `register'", "org/jruby/RubyHash.java:1342:in `each'", "C:/Users/Public/logstash-5.2.1/vendor/bundle/jruby/1.9/gems/logstash-filter-mutate-3.1.3/lib/logstash/filters/mutate.rb:172:in `register'", "C:/Users/Public/logstash-5.2.1/logstash-core/lib/logstash/pipeline.rb:235:in `start_workers'", "org/jruby/RubyArray.java:1613:in `each'", "C:/Users/Public/logstash-5.2.1/logstash-core/lib/logstash/pipeline.rb:235:in `start_workers'", "C:/Users/Public/logstash-5.2.1/logstash-core/lib/logstash/pipeline.rb:188:in `run'", "C:/Users/Public/logstash-5.2.1/logstash-core/lib/logstash/agent.rb:302:in `start_pipeline'"]}

来自日志的单行：

80,17-02-2017 18:28:31,56.000,45.000,0.000,2.000,0.000,44.000,55.000,57.000,50.000

日志中只有几行。

80,17-02-2017 18:28:31,56.000,45.000,0.000,2.000,0.000,44.000,55.000,57.000,50.000

80,17-02-2017 18:28:32,53.000,45.000,0.000,3.000,0.000,54.000,43.000,54.000,43.000

80,17-02-2017 18:28:33,56.000,45.000,0.000,2.000,0.000,45.000,51.000,43.000,50.000

80,17-02-2017 18:28:34,53.000,45.000,0.000,1.000,0.000,42.000,47.000,48.000,48.000

80,17-02-2017 18:28:35,59.000,45.000,0.000,2.000,0.000,45.000,59.000,39.000,48.000

80,17-02-2017 18:28:36,56.000,45.000,0.000,3.000,0.000,44.000,49.000,50.000,50.000

我的过滤器

filter {
     csv {
         columns => ["port", "timestamp", "tempcpuavg", "gputemp", "fanspeed", "gpuusage", "framerate", "tempcpu1", "tempcpu2", "tempcpu3", "tempcpu4"]
                     #80,   17-02-2017 18:28:31,56.000,    45.000,     0.000,      2.000,        0.000,     44.000,      55.000,   57.000,       50.000
         separator => ","
         skip_empty_columns => "true"
         remove_field => ["message"]
     }
     mutate {
         convert => ["port", "integer"]
         convert => ["tempcpuavg", "double"]
         convert => ["gputemp", "double"]
         convert => ["fanspeed", "double"]
         convert => ["gpuusage", "double"]
         convert => ["framerate", "double"]
         convert => ["tempcpu1", "double"]
         convert => ["tempcpu2", "double"]
         convert => ["tempcpu3", "double"]
         convert => ["tempcpu4", "double"]
     }

     date {
         match => [@timestamp", "MM-dd-YYYY HH:mm:ss"]
     }
 }

Answer 1

您的 mutate/convert 过滤器使用了不受支持的数据类型，即 double。 mutate 过滤器的 documentation 表示

Valid conversion targets are: integer, float, string, and boolean.

所以你只需要将所有 double 更改为 float

 mutate {
     convert => ["port", "integer"]
     convert => ["tempcpuavg", "float"]
     convert => ["gputemp", "float"]
     convert => ["fanspeed", "float"]
     convert => ["gpuusage", "float"]
     convert => ["framerate", "float"]
     convert => ["tempcpu1", "float"]
     convert => ["tempcpu2", "float"]
     convert => ["tempcpu3", "float"]
     convert => ["tempcpu4", "float"]
 }

Logstash 管道中止

Logstash Pipeline aborted

csv

elasticsearch

logstash

ruby-debug