按时间间隔生成输出,其中源中没有值

Generating output by time interval where there are no values in source

我有一个消息日志文件,其中包含每条发送的消息一行,时间戳四舍五入到分钟。我可以使用 Logparser 生成关于每分钟消息频率的报告,并输出显示相同信息的图表。这是我用来生成报告的查询:

SELECT  
    QUANTIZE (Date, 60) AS M, 
    COUNT(*) AS Total 
FROM '[LOGFILEPATH]' 
GROUP BY M 
ORDER BY M

这非常适合在生成流量时显示那些分钟的消息流量速率。

但是,我想输出一个图表,该图表还显示消息计数为 0 的分钟间隔 - 在条形图中,条形图为 0分钟。这将向我们显示没有消息流量的时间间隔。

当然,这不在源数据中,因为它没有记录 "no message"。任何想法如何做到这一点?实际上我不在乎它是不是 Logparser - Powershell 是我很乐意使用的其他东西。甚至 Excel,尽管我非常喜欢无需手动干预即可编写脚本的解决方案。

我们是一家 Windows 商店,对开发工具的访问权限有限,因此请不要 Python、bash 等解决方案。

这里有一些样本(真实的,经过混淆处理的)数据,其中包含将近半小时的时间间隔,其中没有流量被记录。

Date,MID,Host,Sender,Recipient,Subject,Last State
2017-10-26 01:56,1078941,smtp3 (10.0.0.156),bounces@example.com,2.O@d.com.au,Notification has been created successfully,Message 1078941 to 2.O@d.com.au received remote SMTP response 'ok'
2017-10-26 01:56,1078938,smtp3 (10.0.0.156),bounces@example.com,a@y.com.au,Notification has been created successfully,Message 1078940 to bounces@example.com received remote SMTP response '2.0.0'
2017-10-26 01:55,4571113,smtp1 (10.0.1.59),bounces@example.com,K@c.com,Notification has been created successfully,Message 4571113 to K@c.com received remote SMTP response '2.0.0'
2017-10-26 01:55,1078936,smtp3 (10.0.0.156),bounces@example.com,G.F@y.com.au,Notification has been created successfully,Message 1078936 to G.F@y.com.au received remote SMTP response 'ok'
2017-10-26 01:54,4571083,smtp1 (10.0.1.59),bounces@example.com,B.H@b.com,Notification has been created successfully,Message 4571083 to B.H@b.com received remote SMTP response 'Mail accepted'.
2017-10-26 01:53,1078927,smtp3 (10.0.0.156),bounces@example.com,S.R@g.com,Notification has been created successfully,Message 1078927 to S.R@g.com received remote SMTP response '2.0.0'
2017-10-26 01:52,4571051,smtp1 (10.0.1.59),bounces@example.com,A.U@r.com.au,Notification has been created successfully,Message 4571051 to A.U@r.com.au received remote SMTP response 'ok'
2017-10-26 01:23,4570598,smtp1 (10.0.1.59),bounces@example.com,T@h.com,Notification has been created successfully,Message 4570598 to T@h.com received remote SMTP response '2.0.0'
2017-10-26 01:23,4570594,smtp1 (10.0.1.59),bounces@example.com,I@k.com.au,Notification has been created successfully,Message 4570594 to I@k.com.au received remote SMTP response 'ok'
2017-10-26 01:22,4570579,smtp1 (10.0.1.59),bounces@example.com,I.C@s.com.au,Notification has been created successfully,Message 4570579 to I.C@s.com.au received remote SMTP response '2.0.0'
2017-10-26 01:22,4570577,smtp1 (10.0.1.59),bounces@example.com,P.M@h.net.au,Notification has been created successfully,Message 4570577 to P.M@h.net.au received remote SMTP response '2.0.0'
2017-10-26 01:22,4570575,smtp1 (10.0.1.59),bounces@example.com,O@h.com.au,Notification has been created successfully,Message 4570575 to O@h.com.au received remote SMTP response '2.0.0'
2017-10-26 01:21,4570571,smtp1 (10.0.1.59),bounces@example.com,O@f.com,Notification has been created successfully,Message 4570571 to O@f.com received remote SMTP response 'ok'
2017-10-26 01:21,4570557,smtp1 (10.0.1.59),bounces@example.com,O.M@m.v.edu.au,Notification has been created successfully,Message 4570557 to O.M@m.v.edu.au received remote SMTP response 'OK'
2017-10-26 01:21,4570549,smtp1 (10.0.1.59),bounces@example.com,A@a.com,Notification has been created successfully,Message 4570550 to bounces@example.com received remote SMTP response '2.0.0'
2017-10-26 01:20,1078803,smtp3 (10.0.0.156),bounces@example.com,M.1@l.com.au,Notification has been created successfully,Message 1078803 to M.1@l.com.au received remote SMTP response '2.0.0'
2017-10-26 01:20,1078802,smtp3 (10.0.0.156),bounces@example.com,B@g.com,New Account,Message 1078802 to B@g.com received remote SMTP response '2.0.0'
2017-10-26 01:20,4570539,smtp1 (10.0.1.59),bounces@example.com,I@v.com.au,Notification has been created successfully,Message 4570539 to I@v.com.au received remote SMTP response 'OK'
2017-10-26 01:19,1078794,smtp3 (10.0.0.156),bounces@example.com,A.T@b.com,Notification has been created successfully,Message 1078794 to A.T@b.com received remote SMTP response 'ok'

如评论中所述,这将是一个多步骤过程,从获取日志文件中涵盖的日期开始:

SELECT MIN(Date) AS MinDate, MAX(Date) as MaxDate 
INTO theDates.csv 
FROM yourlog.log

读取 theDates.csv 中包含的日期,例如使用 Powershell 程序生成与您的日志格式匹配且每分钟恰好包含 1 条记录的日志文件。

然后您可以 运行 您的原始查询稍微调整一下:

SELECT 
   QUANTIZE(Date, 60) as M,
   SUB(COUNT(*), 1) as total
FROM yourlog.log, yourfakelogfile.log
ORDER BY M
GROUP BY M

计数被减 1 的细微调整,以便在没有 activity.

的时间段内得到零

将所有这三个步骤放在一个 powershell 脚本中,您就拥有了一个自动化的、可重复的过程。