logstash RegexpError: invalid char in group name

Question

我一直在尝试使用 logstash.config 中的以下正则表达式从字段中提取部分字符串，

{
  "_index": "logstash-2016.08.09",
  "_type": "log",
  "_id": "AVZvz2ix",
  "_score": null,
  "_source": {
    "message": "function_name~execute||line_no~128||debug_message~id was not found",
    "@version": "1",
    "@timestamp": "2016-08-09T14:57:00.147Z",
    "beat": {
      "hostname": "coredev",
      "name": "coredev"
    },
    "count": 1,
    "fields": null,
    "input_type": "log",
    "offset": 22299196,
    "source": "/project_root/project_1/log/core.log",
    "type": "log",
    "host": "coredev",
    "tags": [
      "beats_input_codec_plain_applied"
    ]
  },
  "fields": {
    "@timestamp": [
      1470754620147
    ]
  },
  "sort": [
    1470754620147
  ]
}

例如，从"source": "/project_root/project_1/log/core.log"中提取core.log。

filter {
  grok {
    match => ["source", "/(?<[@metadata][log_type]>[^/]+)$"]
  }
}

但出现错误，

{:timestamp=>"2016-08-16T10:09:41.352000+0000", :message=>"Pipeline aborted due to error", :exception=>#<RegexpError: invalid char in group name <[@metadata][log_type]>: /\/(?<[@metadata][log_type]>[^\/]+)$/m>, :backtrace=>["org/jruby/RubyRegexp.java:1434:in `initialize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.2/lib/grok-pure.rb:127:in `compile'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-2.0.5/lib/logstash/filters/grok.rb:264:in `register'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-2.0.5/lib/logstash/filters/grok.rb:259:in `register'", "org/jruby/RubyHash.java:1342:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-2.0.5/lib/logstash/filters/grok.rb:255:in `register'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:182:in `start_workers'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:182:in `start_workers'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:136:in `run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/agent.rb:473:in `start_pipeline'"], :level=>:error}

正则表达式有什么问题？

Answer 1

所以你不喜欢 ;-)

无论如何，您需要像这样指定您的自定义模式：

filter {
  grok {
    match => ["source", ".*\/(?<log_type>.*)"]
  }
}

但是请注意，由于 this issue，无法在 grok 正则表达式的命名捕获中指定嵌套字段。

这将捕获 core.log 并将其存储在事件的 log_type 字段中。然后，您可以根据需要将 log_type 移动到 @metadata 字段。

logstash RegexpError: invalid char in group name

logstash RegexpError: invalid char in group name

logstash

logstash-grok

logstash-configuration