Kusto - 帮助编写 KQL 数据透视表

Question

在 IoT 项目中，我们在 Azure 数据资源管理器中收集传感器数据。所有传感器数据都存储在“信号”table 中。为了唯一地识别给定传感器的时间序列，我们这样查询：

Signals
| where TestId == "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3" and SignalName == "Signal1"

我们希望能够将给定 TestId 的所有时间序列从“信号”Table 行转换为列。我一直无法编写实现此目的的 Kusto 查询，希望在此论坛上获得一些帮助。

原样

当前信号 table 架构如下所示：

Timestamp	TestId	SignalName	Value
2021-01-01 12:00:30	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	Signal1	23400
2021-01-01 12:00:30	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	Signal2	0.113
2021-01-01 12:00:30	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	Signal3	77.5
2021-01-01 12:00:31	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	Signal1	23450
2021-01-01 12:00:31	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	Signal2	0.114
2021-01-01 12:00:31	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	Signal3	75.4
2021-01-01 12:00:32	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	Signal1	22450
2021-01-01 12:00:32	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	Signal2	0.113
2021-01-01 12:00:32	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	Signal3	80.05

未来

我希望能够将 Table 转换为以下架构：

Timestamp	TestId	Signal1	Signal2	Signal3
2021-01-01 12:00:30	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	23400	0.113	77.5
2021-01-01 12:00:31	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	23450	0.114	75.4
2021-01-01 12:00:32	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	22450	0.113	80.05

我试过以下查询：

let testId = "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3";
signals
| where TestId == testId 
| where SignalName == "Signal1" or SignalName == "Signal2" or SignalName == "Signal3"
| order by Timestamp desc
| evaluate pivot(SignalName)

但生成的 table 似乎重复了时间戳 - 时间戳被多次表示，并且在其他信号列中插入了默认值“0”：

Timestamp	TestId	Signal1	Signal2	Signal3
2021-01-01 12:00:30	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	23400	0	0
2021-01-01 12:00:30	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	0	0.113	0
2021-01-01 12:00:30	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	0	0	77.5
2021-01-01 12:00:31	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	23450	0	0
2021-01-01 12:00:31	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	0	0.114	0
2021-01-01 12:00:31	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	0	0	75.4
2021-01-01 12:00:32	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	22450	0	0
2021-01-01 12:00:32	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	0	0.113	0
2021-01-01 12:00:32	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	0	0	80.5

我不需要使用 Pivot 运算符进行任何聚合，因为所有信号都应该在完全相同的时间戳上有一个值。

有人可以帮我写一个 KQL 查询吗？
我是否需要在 Azure 数据资源管理器中创建实体化视图才能实现此目的？更新政策或功能？

谢谢

Answer 1

请看下面：

datatable(Timestamp:datetime, TestId:string, SignalName:string, Value:double)
[
    datetime(2021-01-01 12:00:30),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal1",  23400,
    datetime(2021-01-01 12:00:30),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal2", 0.113,
    datetime(2021-01-01 12:00:30),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal3",  77.5,
    datetime(2021-01-01 12:00:31),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal1",  23450,
    datetime(2021-01-01 12:00:31),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal2",  0.114,
    datetime(2021-01-01 12:00:31),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal3",  75.4,
    datetime(2021-01-01 12:00:32),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal1",  22450,
    datetime(2021-01-01 12:00:32),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal2",  0.113,
    datetime(2021-01-01 12:00:32),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal3",  80.05
]
| project Timestamp, TestId, P = pack(SignalName, Value)
| summarize make_bag(P) by Timestamp, TestId
| evaluate bag_unpack(bag_P)

Timestamp	TestId	Signal1	Signal2	Signal3
2021-01-01 12:00:30.0000000	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	23400	0.113	77.5
2021-01-01 12:00:31.0000000	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	23450	0.114	75.4
2021-01-01 12:00:32.0000000	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	22450	0.113	80.05

Answer 2

您需要在数据透视插件中指定聚合函数：

datatable(Timestamp:datetime, TestId:string, SignalName:string, Value:double)
[
    datetime(2021-01-01 12:00:30),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal1",  23400,
    datetime(2021-01-01 12:00:30),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal2", 0.113,
    datetime(2021-01-01 12:00:30),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal3",  77.5,
    datetime(2021-01-01 12:00:31),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal1",  23450,
    datetime(2021-01-01 12:00:31),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal2",  0.114,
    datetime(2021-01-01 12:00:31),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal3",  75.4,
    datetime(2021-01-01 12:00:32),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal1",  22450,
    datetime(2021-01-01 12:00:32),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal2",  0.113,
    datetime(2021-01-01 12:00:32),  "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3", "Signal3",  80.05
]
| evaluate pivot(SignalName, sum(Value))

Timestamp	TestId	Signal1	Signal2	Signal3
2021-01-01 12:00:30.0000000	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	23400	0.113	77.5
2021-01-01 12:00:31.0000000	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	23450	0.114	75.4
2021-01-01 12:00:32.0000000	cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3	22450	0.113	80.05

Kusto - 帮助编写 KQL 数据透视表

Kusto - Help writing KQL Pivot

kql

azure-data-explorer