AWS 将多行日志文件从 CloudWatch 流式传输到 ELK

Question

我们正在将应用程序日志从 CloudWatch 流式传输到 AWS ELK。我们的微服务是用 Java 编写的，所以我只关注那些。记录时的典型 java 异常堆栈跟踪如下所示：

Exception in thread "main" java.lang.NullPointerException
    at com.example.myproject.Book.getTitle(Book.java:16)
    at com.example.myproject.Author.getBookTitles(Author.java:25)
    at com.example.myproject.Bootstrap.main(Bootstrap.java:14)

通常情况下，这将在 ELK 堆栈中逐行提取，这会破坏整个消息。

通常，要将整个堆栈跟踪记录为一条消息，可以在 Logstash 或 Filebeat 中配置 multiline 插件。

知道如何在通过 AWS lambda 将日志文件从 CloudWatch 流式传输到 ELK 时启用多行吗？

Answer 1

事实证明这是 AWS cloudwatch 记录器的已知问题，请参阅 RichardBronosky 在 https://github.com/visionmedia/debug/issues/296

的评论

I did some testing and found that CloudWatch log entries can be made multiline by using \r as the line delimiter. Using either \n (Unix) or \r\n (DOS) line endings will result in separate entries

所以我通过将以下 ExceptionHandler 添加到 Spring Boot RestController 来修复它

@ExceptionHandler(Throwable.class)
void handleUnhandledExceptions(Throwable e, HttpServletResponse response) throws IOException {
    StringWriter buffer = new StringWriter();
    e.printStackTrace(new PrintWriter(buffer));
    log.error(buffer.toString().replace("\n", "\r"));
    response.sendError(HttpStatus.INTERNAL_SERVER_ERROR.value(), e.getMessage());
}

AWS 将多行日志文件从 CloudWatch 流式传输到 ELK

AWS streaming multi-line log files from CloudWatch to ELK

amazon-web-services

amazon-cloudwatch

elastic-stack

amazon-elasticsearch