当日期在日期范围内时加入(Kusto/KQL/Azure 数据资源管理器)
Join when a date is within a date range ( Kusto / KQL / Azure Data Explorer )
在 KQL 中,如何添加连接条件?例如,下面的查询显示了一个连接。我只想在 'code' 列相等且 'date' 介于 StartDate 和 EndDate 之间时加入行。我知道这在 SQL 中是可行的,但还没有在 KQL 中看到有效的示例。
请记住,下面的示例不是实际的数据集。我知道我可以通过过滤 post 加入来达到预期的结果;但是,由于 table B 的大小,这对于实际数据集是不可能的。当我尝试过滤 post 连接时,我 运行 内存不足 (运行away查询)。
let A = datatable(Code:string, StartDate:string, EndDate:string, Rate:string)[
'EUR', '2020-01-01', '2020-01-15', '1.5',
'EUR', '2020-01-15', '2020-01-31', '1.7',
'YEN', '2020-01-01', '2020-01-15', '2.1',
'YEN', '2020-01-15', '2020-01-31', '1.9'];
let B = datatable(Key:string, Code:string, Date:string)[
'1', 'EUR', '2020-01-02',
'2', 'EUR', '2020-01-16',
'3', 'EUR', '2020-01-13',
'4', 'EUR', '2020-01-27',
'5', 'YEN', '2020-01-03',
'6', 'YEN', '2020-01-03', ];
A | join kind=inner B on Code
| project-away *1, StartDate, EndDate
| summarize by Key, Date, Code, Rate
当前结果
Key Date Code Rate
1 2020-01-02 EUR 1.7
1 2020-01-02 EUR 1.5
2 2020-01-16 EUR 1.7
2 2020-01-16 EUR 1.5
3 2020-01-13 EUR 1.7
3 2020-01-13 EUR 1.5
4 2020-01-27 EUR 1.7
4 2020-01-27 EUR 1.5
5 2020-01-03 YEN 1.9
5 2020-01-03 YEN 2.1
6 2020-01-03 YEN 1.9
6 2020-01-03 YEN 2.1
预期结果
Key Date Code Rate
1 2020-01-02 EUR 1.5
2 2020-01-16 EUR 1.7
3 2020-01-13 EUR 1.5
4 2020-01-27 EUR 1.7
5 2020-01-03 YEN 2.1
6 2020-01-03 YEN 2.1
您可以在联接后添加过滤器并使用 Time window join 技术
let lookupWindow = 1d;
let A = datatable(Code:string, StartDate:datetime , EndDate:datetime , Rate:string)[
'EUR', '2020-01-01', '2020-01-15', '1.5',
'EUR', '2020-01-15', '2020-01-31', '1.7',
'YEN', '2020-01-01', '2020-01-15', '2.1',
'YEN', '2020-01-15', '2020-01-31', '1.9'];
let B = datatable(Key:string, Code:string, Date:datetime )[
'1', 'EUR', '2020-01-02',
'2', 'EUR', '2020-01-16',
'3', 'EUR', '2020-01-13',
'4', 'EUR', '2020-01-27',
'5', 'YEN', '2020-01-03',
'6', 'YEN', '2020-01-03', ];
A
| extend TimeKey = range(bin(StartDate, lookupWindow), bin(EndDate,lookupWindow), lookupWindow)
| mv-expand TimeKey to typeof(datetime)
| join kind=inner (B
| extend TimeKey = bin(Date, 1d)
) on Code,TimeKey
| where Date between (StartDate .. EndDate)
| project-away StartDate, EndDate, TimeKey*, Code1
Code
Rate
Key
Date
EUR
1.5
1
2020-01-02 00:00:00.0000000
EUR
1.5
3
2020-01-13 00:00:00.0000000
EUR
1.7
2
2020-01-16 00:00:00.0000000
EUR
1.7
4
2020-01-27 00:00:00.0000000
YEN
2.1
5
2020-01-03 00:00:00.0000000
YEN
2.1
6
2020-01-03 00:00:00.0000000
在 KQL 中,如何添加连接条件?例如,下面的查询显示了一个连接。我只想在 'code' 列相等且 'date' 介于 StartDate 和 EndDate 之间时加入行。我知道这在 SQL 中是可行的,但还没有在 KQL 中看到有效的示例。
请记住,下面的示例不是实际的数据集。我知道我可以通过过滤 post 加入来达到预期的结果;但是,由于 table B 的大小,这对于实际数据集是不可能的。当我尝试过滤 post 连接时,我 运行 内存不足 (运行away查询)。
let A = datatable(Code:string, StartDate:string, EndDate:string, Rate:string)[
'EUR', '2020-01-01', '2020-01-15', '1.5',
'EUR', '2020-01-15', '2020-01-31', '1.7',
'YEN', '2020-01-01', '2020-01-15', '2.1',
'YEN', '2020-01-15', '2020-01-31', '1.9'];
let B = datatable(Key:string, Code:string, Date:string)[
'1', 'EUR', '2020-01-02',
'2', 'EUR', '2020-01-16',
'3', 'EUR', '2020-01-13',
'4', 'EUR', '2020-01-27',
'5', 'YEN', '2020-01-03',
'6', 'YEN', '2020-01-03', ];
A | join kind=inner B on Code
| project-away *1, StartDate, EndDate
| summarize by Key, Date, Code, Rate
当前结果
Key Date Code Rate
1 2020-01-02 EUR 1.7
1 2020-01-02 EUR 1.5
2 2020-01-16 EUR 1.7
2 2020-01-16 EUR 1.5
3 2020-01-13 EUR 1.7
3 2020-01-13 EUR 1.5
4 2020-01-27 EUR 1.7
4 2020-01-27 EUR 1.5
5 2020-01-03 YEN 1.9
5 2020-01-03 YEN 2.1
6 2020-01-03 YEN 1.9
6 2020-01-03 YEN 2.1
预期结果
Key Date Code Rate
1 2020-01-02 EUR 1.5
2 2020-01-16 EUR 1.7
3 2020-01-13 EUR 1.5
4 2020-01-27 EUR 1.7
5 2020-01-03 YEN 2.1
6 2020-01-03 YEN 2.1
您可以在联接后添加过滤器并使用 Time window join 技术
let lookupWindow = 1d;
let A = datatable(Code:string, StartDate:datetime , EndDate:datetime , Rate:string)[
'EUR', '2020-01-01', '2020-01-15', '1.5',
'EUR', '2020-01-15', '2020-01-31', '1.7',
'YEN', '2020-01-01', '2020-01-15', '2.1',
'YEN', '2020-01-15', '2020-01-31', '1.9'];
let B = datatable(Key:string, Code:string, Date:datetime )[
'1', 'EUR', '2020-01-02',
'2', 'EUR', '2020-01-16',
'3', 'EUR', '2020-01-13',
'4', 'EUR', '2020-01-27',
'5', 'YEN', '2020-01-03',
'6', 'YEN', '2020-01-03', ];
A
| extend TimeKey = range(bin(StartDate, lookupWindow), bin(EndDate,lookupWindow), lookupWindow)
| mv-expand TimeKey to typeof(datetime)
| join kind=inner (B
| extend TimeKey = bin(Date, 1d)
) on Code,TimeKey
| where Date between (StartDate .. EndDate)
| project-away StartDate, EndDate, TimeKey*, Code1
Code | Rate | Key | Date |
---|---|---|---|
EUR | 1.5 | 1 | 2020-01-02 00:00:00.0000000 |
EUR | 1.5 | 3 | 2020-01-13 00:00:00.0000000 |
EUR | 1.7 | 2 | 2020-01-16 00:00:00.0000000 |
EUR | 1.7 | 4 | 2020-01-27 00:00:00.0000000 |
YEN | 2.1 | 5 | 2020-01-03 00:00:00.0000000 |
YEN | 2.1 | 6 | 2020-01-03 00:00:00.0000000 |