如何在 Splunk 连接查询中显示两个事件之间的时间差?
How to show the time difference between two events in a Splunk join query?
考虑以下日志数据,
{"level":"info","msg":"Enqueued a recurring task","time":"2022-04-01T18:46:22.854684-07:00","uuid":"4e9fc098-f611-4128-ae0b-2e6b9cb232c8"}
{"level":"info","msg":"Reported a result","path":"/results","time":"2022-04-01T18:46:22.955999-07:00","uuid":"4e9fc098-f611-4128-ae0b-2e6b9cb232c8"}
{"level":"info","msg":"Reported a result","path":"/results","time":"2022-04-01T18:46:23.056295-07:00","uuid":"4e9fc098-f611-4128-ae0b-2e6b9cb232c8"}
{"level":"info","msg":"Enqueued a recurring task","time":"2022-04-01T18:46:23.056376-07:00","uuid":"28e9bea9-5d0c-4dd5-af4f-c22944fc4fcd"}
表示用某个uuid
入队一个循环任务,其结果可以用同一个uuid
多次上报(或者根本不上报)。我对确定任务入队和第一次报告结果之间经过的时间间隔很感兴趣。到目前为止,我可以显示这个外连接的结果 table,
msg="Enqueued a recurring task"
| join type=outer left=L right=R where L.uuid = R.uuid
[ search msg="Reported a result" | dedup uuid sortby +_time]
| fillnull value=""
| table _time, L.msg, L.uuid, R.msg, L.time, R.time
为了方便起见,我想添加一个附加列,其中包含 R.time
和 L.time
之间的差异。据我所知 how to calculate duration between two events Splunk, one way to do this is to use strptime 将这些时间字段转换为时间值,然后确定它们的差异。但是,在导入数据时已经解析了事件的时间(从内置的 _time
字段可以看出)所以这对我来说似乎效率低下。有没有更有效的方法?
更新
即使使用 strptime
的 'inefficient' 方法也被证明是棘手的;我已将 eval
个字段 t
添加到每个搜索中,
msg="Enqueued a recurring task"
| eval t=strptime(time, "%Y-%m-%dT%H:%M:%S.%6N%:z")
| join type=outer left=L right=R where L.uuid = R.uuid
[ search msg="Reported a result"
| dedup uuid sortby +_time
| eval t=strptime(time, "%Y-%m-%dT%H:%M:%S.%6N%:z")]
| fillnull value=""
| table _time, L.msg, L.uuid, R.msg, L.time, R.time, L.t, R.t, R.t-L.t
它似乎正确解析了时间,但差异 R.t-L.t
在 table 中显示为空:
知道为什么它没有显示出差异吗?
更新 2
我已经尝试了 RichG 的两个答案,但它们都会导致 Splunk 出现以下错误:
Error in 'eval' command: Type checking failed. '-' only takes numbers.
(见下面的截图)。我 运行 Splunk 的方式是使用 Docker 容器,
docker run -p 8000:8000 -e "SPLUNK_START_ARGS=--accept-license" -e "SPLUNK_PASSWORD=supersecret" --name splunk splunk/splunk:latest
更新 3
我终于使用 RichG 的更新答案实现了这一点,结合了评估的 strftime
字段和 diff
字段,其中引用了 R.t
和 L.t
:
msg="Enqueued a recurring task"
| eval t=strptime(time, "%Y-%m-%dT%H:%M:%S.%6N%:z")
| join type=outer left=L right=R where L.uuid = R.uuid
[ search msg="Reported a result"
| dedup uuid sortby +_time
| eval t=strptime(time, "%Y-%m-%dT%H:%M:%S.%6N%:z")]
| fillnull value=""
| eval diff='R.t'-'L.t'
| table _time, L.msg, L.uuid, R.msg, L.time, R.time, L.t, R.t, diff
table中的时间差为空,因为table
命令不进行运算。您必须在单独的 eval
中计算差异并将结果显示在 table.
中
index=foo msg="Enqueued a recurring task"
| eval t=strptime(time, "%Y-%m-%dT%H:%M:%S.%6N%:z")
| join type=outer left=L right=R where L.uuid = R.uuid
[ search index=foo msg="Reported a result"
| dedup uuid sortby +_time
| eval t=strptime(time, "%Y-%m-%dT%H:%M:%S.%6N%:z")]
| fillnull value=""
| eval diff='R.t' - 'L.t'
| table _time, L.msg, L.uuid, R.msg, L.time, R.time, L.t, R.t, diff
您应该能够使用每个事件的 _time
字段以避免解析时间字段。
index=foo msg="Enqueued a recurring task"
| join type=outer left=L right=R where L.uuid = R.uuid
[ search index=foo msg="Reported a result"
| dedup uuid sortby +_time ]
| fillnull value=""
| eval diff='R._time' - 'L._time'
| table _time, L.msg, L.uuid, R.msg, L.time, R.time, L.t, R.t, diff
相当有信心类似这样的东西应该有效(不使用 join
:
index=ndx sourtype=srctp uuid=* msg=*
| stats min(_time) as first_time max(_time) as last_time earliest(msg) as first_msg latest(msg) as last_msg by uuid
| eval diff_seconds=last_time-first_time
| eval first_time=strftime(first_time,"%c"), last_time=strftime(last_time,"%c")
此方法假定 _time
已在 sourcetype 的 props.conf
中正确设置,但如果已正确设置,这将一次性为您提供所需内容。
考虑以下日志数据,
{"level":"info","msg":"Enqueued a recurring task","time":"2022-04-01T18:46:22.854684-07:00","uuid":"4e9fc098-f611-4128-ae0b-2e6b9cb232c8"}
{"level":"info","msg":"Reported a result","path":"/results","time":"2022-04-01T18:46:22.955999-07:00","uuid":"4e9fc098-f611-4128-ae0b-2e6b9cb232c8"}
{"level":"info","msg":"Reported a result","path":"/results","time":"2022-04-01T18:46:23.056295-07:00","uuid":"4e9fc098-f611-4128-ae0b-2e6b9cb232c8"}
{"level":"info","msg":"Enqueued a recurring task","time":"2022-04-01T18:46:23.056376-07:00","uuid":"28e9bea9-5d0c-4dd5-af4f-c22944fc4fcd"}
表示用某个uuid
入队一个循环任务,其结果可以用同一个uuid
多次上报(或者根本不上报)。我对确定任务入队和第一次报告结果之间经过的时间间隔很感兴趣。到目前为止,我可以显示这个外连接的结果 table,
msg="Enqueued a recurring task"
| join type=outer left=L right=R where L.uuid = R.uuid
[ search msg="Reported a result" | dedup uuid sortby +_time]
| fillnull value=""
| table _time, L.msg, L.uuid, R.msg, L.time, R.time
为了方便起见,我想添加一个附加列,其中包含 R.time
和 L.time
之间的差异。据我所知 how to calculate duration between two events Splunk, one way to do this is to use strptime 将这些时间字段转换为时间值,然后确定它们的差异。但是,在导入数据时已经解析了事件的时间(从内置的 _time
字段可以看出)所以这对我来说似乎效率低下。有没有更有效的方法?
更新
即使使用 strptime
的 'inefficient' 方法也被证明是棘手的;我已将 eval
个字段 t
添加到每个搜索中,
msg="Enqueued a recurring task"
| eval t=strptime(time, "%Y-%m-%dT%H:%M:%S.%6N%:z")
| join type=outer left=L right=R where L.uuid = R.uuid
[ search msg="Reported a result"
| dedup uuid sortby +_time
| eval t=strptime(time, "%Y-%m-%dT%H:%M:%S.%6N%:z")]
| fillnull value=""
| table _time, L.msg, L.uuid, R.msg, L.time, R.time, L.t, R.t, R.t-L.t
它似乎正确解析了时间,但差异 R.t-L.t
在 table 中显示为空:
更新 2
我已经尝试了 RichG 的两个答案,但它们都会导致 Splunk 出现以下错误:
Error in 'eval' command: Type checking failed. '-' only takes numbers.
(见下面的截图)。我 运行 Splunk 的方式是使用 Docker 容器,
docker run -p 8000:8000 -e "SPLUNK_START_ARGS=--accept-license" -e "SPLUNK_PASSWORD=supersecret" --name splunk splunk/splunk:latest
更新 3
我终于使用 RichG 的更新答案实现了这一点,结合了评估的 strftime
字段和 diff
字段,其中引用了 R.t
和 L.t
:
msg="Enqueued a recurring task"
| eval t=strptime(time, "%Y-%m-%dT%H:%M:%S.%6N%:z")
| join type=outer left=L right=R where L.uuid = R.uuid
[ search msg="Reported a result"
| dedup uuid sortby +_time
| eval t=strptime(time, "%Y-%m-%dT%H:%M:%S.%6N%:z")]
| fillnull value=""
| eval diff='R.t'-'L.t'
| table _time, L.msg, L.uuid, R.msg, L.time, R.time, L.t, R.t, diff
table中的时间差为空,因为table
命令不进行运算。您必须在单独的 eval
中计算差异并将结果显示在 table.
index=foo msg="Enqueued a recurring task"
| eval t=strptime(time, "%Y-%m-%dT%H:%M:%S.%6N%:z")
| join type=outer left=L right=R where L.uuid = R.uuid
[ search index=foo msg="Reported a result"
| dedup uuid sortby +_time
| eval t=strptime(time, "%Y-%m-%dT%H:%M:%S.%6N%:z")]
| fillnull value=""
| eval diff='R.t' - 'L.t'
| table _time, L.msg, L.uuid, R.msg, L.time, R.time, L.t, R.t, diff
您应该能够使用每个事件的 _time
字段以避免解析时间字段。
index=foo msg="Enqueued a recurring task"
| join type=outer left=L right=R where L.uuid = R.uuid
[ search index=foo msg="Reported a result"
| dedup uuid sortby +_time ]
| fillnull value=""
| eval diff='R._time' - 'L._time'
| table _time, L.msg, L.uuid, R.msg, L.time, R.time, L.t, R.t, diff
相当有信心类似这样的东西应该有效(不使用 join
:
index=ndx sourtype=srctp uuid=* msg=*
| stats min(_time) as first_time max(_time) as last_time earliest(msg) as first_msg latest(msg) as last_msg by uuid
| eval diff_seconds=last_time-first_time
| eval first_time=strftime(first_time,"%c"), last_time=strftime(last_time,"%c")
此方法假定 _time
已在 sourcetype 的 props.conf
中正确设置,但如果已正确设置,这将一次性为您提供所需内容。