Elastic Beanstalk：Cloudwatch 代理随机停止流式传输日志

Question

我有一个通过 ElasticBeanstalk 部署的简单 nodejs (express) 应用程序，我在配置中启用了 cloudwatch 日志，日志开始流式传输。问题是，它只是随机停止。我四处搜索并遇到了答案，在它之后重新启动流式传输但一段时间后它再次挂断，我必须重新启动它。我如何首先防止它发生？

我什至尝试手动提供配置。这是我放在 .ebextensions 目录下的 .config 文件。之后没有区别，从 here, direct file link.

中获取了这个样本

packages:
  yum:
    awslogs: []

files:
  "/etc/awslogs/awscli.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [plugins]
      cwlogs = cwlogs
      [default]
      region = `{"Ref":"AWS::Region"}`

  "/etc/awslogs/awslogs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [general]
      state_file = /var/lib/awslogs/agent-state
      time_zone = UTC
      datetime_format = %Y-%m-%d %H:%M:%S
      log_stream_name = {instance_id}

  "/etc/awslogs/config/logs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [/var/log/eb-activity.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/var/log/eb-activity.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/eb-activity.log
      time_zone = UTC
      datetime_format = %Y-%m-%d %H:%M:%S
      multi_line_start_pattern = {datetime_format}

      [/var/log/nodejs/nodejs.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/var/log/nodejs/nodejs.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/nodejs/nodejs.log
      time_zone = UTC
      datetime_format = %Y-%m-%d %H:%M:%S
      multi_line_start_pattern = {datetime_format}

      [/var/log/nginx/error.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/var/log/nginx/error.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/nginx/error.log
      time_zone = UTC
      datetime_format = %Y-%m-%d %H:%M:%S
      multi_line_start_pattern = {datetime_format}

      [/var/log/nginx/access.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/var/log/nginx/access.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/nginx/access.log
      time_zone = UTC
      datetime_format = %Y-%m-%d %H:%M:%S
      multi_line_start_pattern = {datetime_format}

      [/var/log/httpd/error.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/var/log/httpd/error.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/httpd/error.log
      time_zone = UTC
      datetime_format = %Y-%m-%d %H:%M:%S
      multi_line_start_pattern = {datetime_format}

      [/var/log/httpd/access.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/var/log/httpd/access.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/httpd/access.log
      time_zone = UTC
      datetime_format = %Y-%m-%d %H:%M:%S
      multi_line_start_pattern = {datetime_format}

commands:
  "01":
    command: chkconfig awslogs on
  "02":
    command: service awslogs restart

在 cloudwatch 代理参考中 FAQ 我看到它说：

If the timestamp of log event is more than 2 hours in future, the log event is skipped

在我的 var/log/awslogs.log 中，我确实注意到这个错误：

someDateTime - cwlogs.push.batch - WARNING - xxxxx - Thread-4 - Skip event: {'timestamp': xxxxxxxxxx, 'start_position': xxxxxxx, 'end_position': xxxxxxxxx}, reason: timestamp is more than 2 hours in future.

如何防止这种情况发生？

Answer 1

这个问题发布有点晚，但我在 AWS 支持的帮助下解决了这个问题。我们的负载平衡器配置为 ping 服务器以进行健康检查，该服务器正在生成特定日志，例如“请求中未找到令牌”。健康检查已通过，因为它被配置为检查它但不断记录相同的字符串导致 CW 代理在为下一个周期创建哈希时无法正常工作。

Elastic Beanstalk：Cloudwatch 代理随机停止流式传输日志

ElasticBeanstalk: Cloudwatch agent stops streaming logs randomely

amazon-web-services

node.js

amazon-cloudwatch

amazon-elastic-beanstalk

amazon-cloudwatchlogs