在 AWS cloudwatch insights 中解析消息后分组

Question

我有如下消息，以下消息是其中一条消息（有这么多JSON格式与此完全无关）

request body to the server {'sender': '65ddd20eac244AAe619383e4d8cb558834', 'message': 'hello'}

我想根据包含在 JSON 中的发件人（字母数字值）对这些邮件进行分组。

Answer 1

你可以使用 filter.

fields @timestamp, @message
| filter @message like "65ddd20eac244AAe619383e4d8cb558834"
| sort @timestamp desc
| limit 20

它将过滤所有由 65ddd20eac244AAe619383e4d8cb558834 发送的消息，限制为 20 条。

更新：

假设 JSON 日志格式是这样的

{
    "sender": "65ddd20eac244AAe619383e4d8cb558835",
    "message": "Hi"
}

现在我想统计来自65ddd20eac244AAe619383e4d8cb558835

的消息数量

how many messages are coming from each user?

如此简单，您可以运行查询

stats count(sender) by sender |
# To filter only message the contain sender, to avoid lambda default logs
filter @message like "sender"

如果您也想查看消息，请稍微修改一下查询

stats count(*) by sender, message |

filter @message like "sender"

这里 @message 指的是整个索引，其中 message 指的是 JSON 对象 message.

count_distinct

Returns the number of unique values for the field. If the field has very high cardinality (contains many unique values), the value returned by count_distinct is just an approximation.

how many distinct users in the selected interval?

它将在 3hr 间隔

中列出不同的用户

 stats count_distinct(sender) as distinct_sender by bin(3hr) as interval

Answer 2

CloudWatch Logs Insights 查询：

fields @message |
filter @message like 'request body to the server' |
parse @message "'sender': '*', 'message'" as sender |
stats count(*) by sender

查询结果：

-------------------------------------------------
|               sender               | count(*) |
|------------------------------------|----------|
| 65ddd20eac244AAe619383e4d8cb558834 |     4    |
| 55ddd20eac244AAe619383e4d8cb558834 |     3    |
-------------------------------------------------

截图：

在 AWS cloudwatch insights 中解析消息后分组

Group By after parsing a message in AWS cloudwatch insights

amazon-web-services

amazon-cloudwatch

amazon-cloudwatchlogs