.net 日志从 filebeat 到 logstash 多行正确格式

Question

我正在尝试从 .net 应用程序中提取日志。我在将日志推送到 logstash 服务器的节点上安装了 filebeat。

日志文件：

2020-06-19 00:00:16.421 +02:00 [Error] [Band.Account.HealthCheckService] [2HB0AJ9Q9AI2O:00000001] The operation was canceled.
System.Threading.Tasks.TaskCanceledException: The operation was canceled. ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled. ---> System.Net.Sockets.SocketException: Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.Security.SslStreamInternal
...

filebeat.yml 配置：

filebeat.prospectors:
- type: log
  enabled: true
  paths:
    - /var/log/log*
  multiline.pattern: '^[[:space:]]'
  multiline.negate: false
  multiline.match: after
  fields_under_root: true
  fields:
    type: Band-account-log
  fields_under_root: true
  exclude_files: ['.gz$']
logging.level: info
output.logstash:
  hosts: ["elk.Band.net"]
  ssl.certificate_authorities: "/etc/pki/tls/certs/logstash-forwarder.crt"

它并没有真正从日志文件中获取所有行，并且时间戳在 kibana 中看起来不对。不太确定我需要修改什么。

filebeat 版本 6.2.4 (amd64)，libbeat 6.2.4*

Answer 1

我设法使用下面的 conf 使其工作：

filebeat.yml:

filebeat.prospectors:
- type: log
  enabled: true
  paths:
    - /var/log/log*
  multiline.pattern: '([12]\d{3}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01]))'
  multiline.negate: true
  multiline.match: after
  fields_under_root: true
  fields:
    type: pirelli-account-log
  fields_under_root: true
  exclude_files: ['.gz$']
logging.level: info
output.logstash:
  hosts: ["elk.server.example"]
  ssl.certificate_authorities: "/etc/pki/tls/certs/logstash-forwarder.crt"

现在日志格式正确。

.net 日志从 filebeat 到 logstash 多行正确格式

.net logs from filebeat to logstash multiline proper format

logging

logstash

kibana

filebeat

elk