KDB:如何从分区 Table 中删除行
KDB: How to Delete rows from Partitioned Table
我使用以下查询从分区 table 中删除行,但它不起作用。在分区 table 中删除行的方法是什么?
delete from SecurityLoan where lender=`SCOTIA, date in inDays, portfolio in portfoliolist
请注意 inDays
和 portfoliolist
是列表
遗憾的是,您不能直接对分区数据库使用删除。
要完全删除一行,您必须再次读取、修改和写入所有数据。
有关如何实现此目的的示例,请参见 wiki:
http://code.kx.com/wiki/JB:KdbplusForMortals/partitioned_tables#1.3.5_Modifying_Partitioned_Tables
谢谢,
肖恩
这里有一个稍微不同的方法,re-indexes 分区中的列到您要保留在该列中的新索引列表。
它仍然遵循读取列、修改然后将其重置回磁盘的相同语义,只是使用了稍微不同的方法。但是,通过这种方式,您只需使用 qsql 查询即可获取要删除的索引。然后它获取分区中的完整索引列表,并针对初始列表运行 'except',从而生成您实际想要保留的索引。
当您只想从 database/table 中删除 sql 查询的内容时,它会变得强大(就像您的情况一样)。
// I've commented this function as much as possible to break it down and explain the approach
// db is where the database lives (hsym)
// qry is the qsql query (string)
q)delFromDisk:{[db;qry]
// grab the tree from the query
q:parse qry;
// cache partition counts
.Q.cn `. t:q 1;
// grab i by partition for your qry using the where clause
d:?[t;raze q 2;{x!x}1#f:.Q.pf;enlist[`delis]!1#`i];
// grab full indice list for each partition
a:1!flip (f,`allis)!(`. f;til each .Q.pn t);
// run except on full indice list and your query's indice list
r:update newis:allis except'delis from a,'d;
// grab columns except partition domain
c:cols[t] except .Q.pf;
// grab partitions that actually need modifications and make them dir handles
p:update dirs:.Q.par[db;;t] each p[.Q.pf] from p:0!select from r where not allis~'newis;
// apply on disk to directory handle (x), on column (y), to new indices (z)
m:{@[x;y;@;z]};
// grab params from p
pa:`dirs`c`newis#p cross ([]c);
// modify each column in a partition, one partition at a time
m .' value each pa
};
// test data/table
q)portfolio:`one`two`three`four`five;
q)lender:`user1`user2`user3`user4;
q)n:5;
// set to disk in date partitioned format
q)`:./2017.01.01/secLoan/ set .Q.en[`:./] ([]lender:n?lender;portfolio:n?portfolio);
q)`:./2017.01.02/secLoan/ set .Q.en[`:./] ([]lender:n?lender;portfolio:n?portfolio);
// load db
q)\l .
// lets say we want to delete from secLoan where lender in `user3 and portfolio in `one`two`three
// please note, this query does not have a date constraint, so it may be an inefficient query if you where-clause produces large results. Once happy with the util as a whole, it can be re-jigged to select+delete per partition
q)select from secLoan where lender in `user3,portfolio in `one`two`three
date lender portfolio
---------------------------
2017.01.01 user3 one
2017.01.01 user3 two
2017.01.02 user3 one
// 3 rows need deleted, 2 from first partition, 1 from second partition
// 10 rows exist
q)count secLoan
10
// run delete function
q)delFromDisk[`:.;"select from secLoan where lender in `user3,portfolio in `one`two`three"];
// reload to see diffs
q)\l .
q)count secLoan
7
// rows deleted
q)secLoan
date lender portfolio
---------------------------
2017.01.01 user2 five
2017.01.01 user4 three
2017.01.01 user2 three
2017.01.02 user2 five
2017.01.02 user2 two
2017.01.02 user4 three
2017.01.02 user1 five
// PS - can accept a delete qsql query as all the function does is look at the where clause
// delFromDisk[`:.;"delete from secLoan where lender in `user3,portfolio in `one`two`three"]
我使用以下查询从分区 table 中删除行,但它不起作用。在分区 table 中删除行的方法是什么?
delete from SecurityLoan where lender=`SCOTIA, date in inDays, portfolio in portfoliolist
请注意 inDays
和 portfoliolist
是列表
遗憾的是,您不能直接对分区数据库使用删除。
要完全删除一行,您必须再次读取、修改和写入所有数据。
有关如何实现此目的的示例,请参见 wiki:
http://code.kx.com/wiki/JB:KdbplusForMortals/partitioned_tables#1.3.5_Modifying_Partitioned_Tables
谢谢, 肖恩
这里有一个稍微不同的方法,re-indexes 分区中的列到您要保留在该列中的新索引列表。
它仍然遵循读取列、修改然后将其重置回磁盘的相同语义,只是使用了稍微不同的方法。但是,通过这种方式,您只需使用 qsql 查询即可获取要删除的索引。然后它获取分区中的完整索引列表,并针对初始列表运行 'except',从而生成您实际想要保留的索引。
当您只想从 database/table 中删除 sql 查询的内容时,它会变得强大(就像您的情况一样)。
// I've commented this function as much as possible to break it down and explain the approach
// db is where the database lives (hsym)
// qry is the qsql query (string)
q)delFromDisk:{[db;qry]
// grab the tree from the query
q:parse qry;
// cache partition counts
.Q.cn `. t:q 1;
// grab i by partition for your qry using the where clause
d:?[t;raze q 2;{x!x}1#f:.Q.pf;enlist[`delis]!1#`i];
// grab full indice list for each partition
a:1!flip (f,`allis)!(`. f;til each .Q.pn t);
// run except on full indice list and your query's indice list
r:update newis:allis except'delis from a,'d;
// grab columns except partition domain
c:cols[t] except .Q.pf;
// grab partitions that actually need modifications and make them dir handles
p:update dirs:.Q.par[db;;t] each p[.Q.pf] from p:0!select from r where not allis~'newis;
// apply on disk to directory handle (x), on column (y), to new indices (z)
m:{@[x;y;@;z]};
// grab params from p
pa:`dirs`c`newis#p cross ([]c);
// modify each column in a partition, one partition at a time
m .' value each pa
};
// test data/table
q)portfolio:`one`two`three`four`five;
q)lender:`user1`user2`user3`user4;
q)n:5;
// set to disk in date partitioned format
q)`:./2017.01.01/secLoan/ set .Q.en[`:./] ([]lender:n?lender;portfolio:n?portfolio);
q)`:./2017.01.02/secLoan/ set .Q.en[`:./] ([]lender:n?lender;portfolio:n?portfolio);
// load db
q)\l .
// lets say we want to delete from secLoan where lender in `user3 and portfolio in `one`two`three
// please note, this query does not have a date constraint, so it may be an inefficient query if you where-clause produces large results. Once happy with the util as a whole, it can be re-jigged to select+delete per partition
q)select from secLoan where lender in `user3,portfolio in `one`two`three
date lender portfolio
---------------------------
2017.01.01 user3 one
2017.01.01 user3 two
2017.01.02 user3 one
// 3 rows need deleted, 2 from first partition, 1 from second partition
// 10 rows exist
q)count secLoan
10
// run delete function
q)delFromDisk[`:.;"select from secLoan where lender in `user3,portfolio in `one`two`three"];
// reload to see diffs
q)\l .
q)count secLoan
7
// rows deleted
q)secLoan
date lender portfolio
---------------------------
2017.01.01 user2 five
2017.01.01 user4 three
2017.01.01 user2 three
2017.01.02 user2 five
2017.01.02 user2 two
2017.01.02 user4 three
2017.01.02 user1 five
// PS - can accept a delete qsql query as all the function does is look at the where clause
// delFromDisk[`:.;"delete from secLoan where lender in `user3,portfolio in `one`two`three"]