Azure 流分析作业将两个事件中心之间不匹配记录的输出延迟 1 分钟

Question

谁能帮帮我，为什么不匹配的记录延迟正好1分钟，而匹配的记录立即写入博客存储容器。

有什么方法可以避免延迟，即使 eventA 与其他 eventB 不匹配（作为我的下游系统会在我的用例中处理）

select b.eventenqueuedutctime as btime,a.Id,a.SysTime,a.UTCTime
,b.Id as BId,b.SysTime as BSysTime 
into outputStorage -- to blob storage (container)
from eventA a TIMESTAMP BY eventenqueuedutctime
left outer join eventB b TIMESTAMP BY eventenqueuedutctime
on a.id = b.id
and datediff(minute,b,a) between 0 and 180 -- join with last 3 hours of eventB data

下面是输出，但请查看最后一行 (Id:99) currentTime:T19:42:13.1690000Z 与前 4 行相比延迟了 1 分钟（ currentTime:T19:41:13.1690000Z)

仅供参考，通过 Json 序列化

通过 EventDataBatch 一次发送所有 eventA Id (2,4,1,101,99)

{"btime":"2020-11-03T17:00:50.6360000Z","Id":2,"SysTime":"2020-11-03T11:41:12.860466-08:00","UTCTime":"2020-11-03T19:41:12.8604646Z","BId":2,"BSysTime":"2020-11-03T09:00:49.6751336-08:00","fullname":"cc","currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":"2020-11-03T17:00:50.6360000Z","Id":4,"SysTime":"2020-11-03T11:41:12.8605138-08:00","UTCTime":"2020-11-03T19:41:12.8605135Z","BId":4,"BSysTime":"2020-11-03T09:00:49.6751371-08:00","fullname":null,"currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":"2020-11-03T17:00:50.6360000Z","Id":1,"SysTime":"2020-11-03T11:41:12.8605561-08:00","UTCTime":"2020-11-03T19:41:12.8605559Z","BId":1,"BSysTime":"2020-11-03T09:00:49.6749841-08:00","fullname":"test","currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":"2020-11-03T19:39:04.0100000Z","Id":101,"SysTime":"2020-11-03T11:41:12.860598-08:00","UTCTime":"2020-11-03T19:41:12.8605978Z","BId":101,"BSysTime":"2020-11-03T11:39:03.7462454-08:00","fullname":"test-101","currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":null,"Id":99,"SysTime":"2020-11-03T11:41:12.860322-08:00","UTCTime":"2020-11-03T19:41:12.8602803Z","BId":null,"BSysTime":null,"fullname":null,"currentTime":"2020-11-03T19:42:13.1690000Z"}

Answer 1

这是因为您将 JOIN 与 DATEDIFF 结合使用。

The use of temporal joins, such as JOIN with DATEDIFF:

Matches generate as soon as both sides of the matched events arrive.

Data that lacks a match, like LEFT OUTER JOIN, is generated at the end of the DATEDIFF window, for each event on the left side.

更多详情，可以参考https://docs.microsoft.com/en-us/azure/stream-analytics/stream-analytics-troubleshoot-output#the-first-output-is-delayed。

Azure 流分析作业将两个事件中心之间不匹配记录的输出延迟 1 分钟

Azure Stream Analytics Job delayed output of not matching records by 1 minutes between two event hubs

sql

azure-eventhub

azure-stream-analytics

azure-blob-storage