我如何在 sqlite table 中找到差距?
How do I find gap in sqlite table?
我有一个 sqlite table,以毫秒为单位的时间戳作为主键,每行之间应该相隔 1 秒或 1000。有时我的数据记录器熄灭,table 中没有当时的数据。如何使用 SQL 语句找到间隙?我知道基于游标的解决方案是可能的。
table = PVT
TS
1119636081000
1119636082000
1119636083000
1119636084000
1119636085000
------gap------
1119636090000
1119636091000
像这样(假设 PVT.TS 是您的列名):
SELECT * FROM 'table' WHERE PVT.TS ISNULL;
或
SELECT * FROM 'table' WHERE PVT.TS IS NULL;
如果您的收藏家实际上输入的是空白条目,您可能需要
WHERE PVT.TS = ''
或
where ifnull(some_column, '') = ''
这可能有效。假设 table 名字是 "tstamps",
select a.ts
from tstamps a
where not exists
(select b.ts
from tstamps b
where b.ts = a.ts+1000)
and exists
(select c.ts
from tstamps c
where c.ts = a.ts+2000)
另一种方式
select a.ts
from tstamps a
where not exists
(select b.ts
from tstamps b
where b.ts = a.ts+1000)
and a.ts <
(select max(c.ts)
from tstamps c
)
使用减号运算符。我不确定,这些查询中哪个查询的性能更好。
select ts+1000
from pvt
where ts != (select max(ts) from pvt)
minus
select ts
from pvt
where ts != (select min(ts) from pvt)
在撰写本文时,SQLite 不支持 window 函数,例如 LAG(TS) OVER (ORDER BY TS ASC)
或 LEAD() OVER
,这很容易为您提供之前和之后的 TS
值,分别
所以,你需要自己动手:
sqlite> .mode col
sqlite> .width 14 14 14
sqlite> SELECT PVT.TS AS measurement,
prev.TS AS prev,
next.TS AS next
FROM PVT
LEFT JOIN PVT next ON PVT.TS = (next.TS - 1000)
LEFT JOIN PVT prev ON PVT.TS = (prev.TS + 1000);
这会给你这样的东西(我使用了不同的数据,你会看到):
-- measurement prev next
------------- ------------- -------------
1119636081000 1119636082000 -- gap (no previous at all)
1119636082000 1119636081000 1119636083000
1119636083000 1119636082000 1119636084000
1119636084000 1119636083000 1119636085000
1119636085000 1119636084000 -- gap (no next offset 1000)
1119636088000 1119636089000 -- gap (no previous offset 1000)
1119636089000 1119636088000 -- gap (no next at all)
您也可以始终将该查询限制为仅那些记录 WHERE prev.TS IS NULL OR next.TS is NULL
。
创建一个至少有 86400 行的计数 table(一天每秒一个):
create table Tally(n integer primary key not null);
insert into Tally(n) values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
insert into Tally(n) select null from tally n1 , tally n2, tally n3, tally n4, tally n5;
将您的 PVT table 加入到当天的转换记录中:
select 1119636081000 + tally.n*1000 as Expected, pvt.ts from tally left join pvt on pvt.ts = 1119636081000 + tally.n*1000 where tally.n <= 86400 limit 15;
鉴于 table 我填充了您的示例数据,我将其作为输出:
Expected TS
------------- -------------
1119636081000 1119636081000
1119636082000 1119636082000
1119636083000 1119636083000
1119636084000 1119636084000
1119636085000 1119636085000
1119636086000
1119636087000
1119636088000
1119636089000
1119636090000 1119636090000
1119636091000 1119636091000
1119636092000
1119636093000
1119636094000
1119636095000
如果您随后在 PVT.TS 为空的位置进行额外过滤,您应该会得到缺失值:
select 1119636081000 + tally.n*1000 as Expected, pvt.ts from tally left join pvt on pvt.ts = 1119636081000 + tally.n*1000 where tally.n <= 86400 and PVT.ts is null limit 15;
Expected TS
------------- ----------
1119636086000
1119636087000
1119636088000
1119636089000
1119636092000
1119636093000
1119636094000
1119636095000
1119636096000
1119636097000
1119636098000
1119636099000
1119636100000
1119636101000
1119636102000
注意:我使用限制 15 来让自己在控制台前保持清醒
我有一个 sqlite table,以毫秒为单位的时间戳作为主键,每行之间应该相隔 1 秒或 1000。有时我的数据记录器熄灭,table 中没有当时的数据。如何使用 SQL 语句找到间隙?我知道基于游标的解决方案是可能的。
table = PVT
TS
1119636081000
1119636082000
1119636083000
1119636084000
1119636085000
------gap------
1119636090000
1119636091000
像这样(假设 PVT.TS 是您的列名):
SELECT * FROM 'table' WHERE PVT.TS ISNULL;
或
SELECT * FROM 'table' WHERE PVT.TS IS NULL;
如果您的收藏家实际上输入的是空白条目,您可能需要
WHERE PVT.TS = ''
或
where ifnull(some_column, '') = ''
这可能有效。假设 table 名字是 "tstamps",
select a.ts
from tstamps a
where not exists
(select b.ts
from tstamps b
where b.ts = a.ts+1000)
and exists
(select c.ts
from tstamps c
where c.ts = a.ts+2000)
另一种方式
select a.ts
from tstamps a
where not exists
(select b.ts
from tstamps b
where b.ts = a.ts+1000)
and a.ts <
(select max(c.ts)
from tstamps c
)
使用减号运算符。我不确定,这些查询中哪个查询的性能更好。
select ts+1000
from pvt
where ts != (select max(ts) from pvt)
minus
select ts
from pvt
where ts != (select min(ts) from pvt)
在撰写本文时,SQLite 不支持 window 函数,例如 LAG(TS) OVER (ORDER BY TS ASC)
或 LEAD() OVER
,这很容易为您提供之前和之后的 TS
值,分别
所以,你需要自己动手:
sqlite> .mode col
sqlite> .width 14 14 14
sqlite> SELECT PVT.TS AS measurement,
prev.TS AS prev,
next.TS AS next
FROM PVT
LEFT JOIN PVT next ON PVT.TS = (next.TS - 1000)
LEFT JOIN PVT prev ON PVT.TS = (prev.TS + 1000);
这会给你这样的东西(我使用了不同的数据,你会看到):
-- measurement prev next
------------- ------------- -------------
1119636081000 1119636082000 -- gap (no previous at all)
1119636082000 1119636081000 1119636083000
1119636083000 1119636082000 1119636084000
1119636084000 1119636083000 1119636085000
1119636085000 1119636084000 -- gap (no next offset 1000)
1119636088000 1119636089000 -- gap (no previous offset 1000)
1119636089000 1119636088000 -- gap (no next at all)
您也可以始终将该查询限制为仅那些记录 WHERE prev.TS IS NULL OR next.TS is NULL
。
创建一个至少有 86400 行的计数 table(一天每秒一个):
create table Tally(n integer primary key not null);
insert into Tally(n) values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
insert into Tally(n) select null from tally n1 , tally n2, tally n3, tally n4, tally n5;
将您的 PVT table 加入到当天的转换记录中:
select 1119636081000 + tally.n*1000 as Expected, pvt.ts from tally left join pvt on pvt.ts = 1119636081000 + tally.n*1000 where tally.n <= 86400 limit 15;
鉴于 table 我填充了您的示例数据,我将其作为输出:
Expected TS
------------- -------------
1119636081000 1119636081000
1119636082000 1119636082000
1119636083000 1119636083000
1119636084000 1119636084000
1119636085000 1119636085000
1119636086000
1119636087000
1119636088000
1119636089000
1119636090000 1119636090000
1119636091000 1119636091000
1119636092000
1119636093000
1119636094000
1119636095000
如果您随后在 PVT.TS 为空的位置进行额外过滤,您应该会得到缺失值:
select 1119636081000 + tally.n*1000 as Expected, pvt.ts from tally left join pvt on pvt.ts = 1119636081000 + tally.n*1000 where tally.n <= 86400 and PVT.ts is null limit 15;
Expected TS
------------- ----------
1119636086000
1119636087000
1119636088000
1119636089000
1119636092000
1119636093000
1119636094000
1119636095000
1119636096000
1119636097000
1119636098000
1119636099000
1119636100000
1119636101000
1119636102000
注意:我使用限制 15 来让自己在控制台前保持清醒