为什么 Postgres 多列分区修剪不比这更聪明？

Question

在 Postgres (v12) 中，我们有一些按“国家/地区”和“部门”分区的大型 tables，但发现修剪行为仅适用于识别精确值的查询对于每个，并且当为其中一个指定 2 个或更多值时变得次优。此外，“国家”列在修剪逻辑中以某种方式受到青睐，大概是因为它是分区键中的第一列...

示例

CREATE TABLE part.partitioned_table
(
    country character varying NOT NULL,
    sector character varying NOT NULL,
    a_value integer NOT NULL,
    other_value integer,
    CONSTRAINT partitioned_table_pkey PRIMARY KEY (country, sector, a_value)
) PARTITION BY RANGE (country, sector, a_value);

CREATE TABLE part.partitioned_table_gb_alpha PARTITION OF part.partitioned_table
    FOR VALUES FROM ('GB', 'ALPHA', MINVALUE) TO ('GB', 'ALPHA', MAXVALUE);
CREATE TABLE part.partitioned_table_gb_beta PARTITION OF part.partitioned_table
    FOR VALUES FROM ('GB', 'BETA', MINVALUE) TO ('GB', 'BETA', MAXVALUE);
CREATE TABLE part.partitioned_table_gb_gamma PARTITION OF part.partitioned_table
    FOR VALUES FROM ('GB', 'GAMMA', MINVALUE) TO ('GB', 'GAMMA', MAXVALUE);
CREATE TABLE part.partitioned_table_fr_alpha PARTITION OF part.partitioned_table
    FOR VALUES FROM ('FR', 'ALPHA', MINVALUE) TO ('FR', 'ALPHA', MAXVALUE);
CREATE TABLE part.partitioned_table_fr_beta PARTITION OF part.partitioned_table
    FOR VALUES FROM ('FR', 'BETA', MINVALUE) TO ('FR', 'BETA', MAXVALUE);
CREATE TABLE part.partitioned_table_fr_gamma PARTITION OF part.partitioned_table
    FOR VALUES FROM ('FR', 'GAMMA', MINVALUE) TO ('FR', 'GAMMA', MAXVALUE);
CREATE TABLE part.partitioned_table_de_alpha PARTITION OF part.partitioned_table
    FOR VALUES FROM ('DE', 'ALPHA', MINVALUE) TO ('DE', 'ALPHA', MAXVALUE);
CREATE TABLE part.partitioned_table_de_beta PARTITION OF part.partitioned_table
    FOR VALUES FROM ('DE', 'BETA', MINVALUE) TO ('DE', 'BETA', MAXVALUE);
CREATE TABLE part.partitioned_table_de_gamma PARTITION OF part.partitioned_table
    FOR VALUES FROM ('DE', 'GAMMA', MINVALUE) TO ('DE', 'GAMMA', MAXVALUE);

INSERT INTO part.partitioned_table(country, sector, a_value, other_value) VALUES
    ('GB', 'ALPHA', 10, 1000),
    ('GB', 'BETA', 10, 1000),
    ('GB', 'GAMMA', 10, 1000),
    ('FR', 'ALPHA', 10, 1000),
    ('FR', 'BETA', 10, 1000),
    ('FR', 'GAMMA', 10, 1000),
    ('DE', 'ALPHA', 10, 1000),
    ('DE', 'BETA', 10, 1000),
    ('DE', 'GAMMA', 10, 1000);

-- query plan for this statement shows that only a single partition is scanned as expected (partitioned_table_gb_beta)
EXPLAIN ANALYSE SELECT * FROM part.partitioned_table WHERE country = 'GB' and sector = 'BETA';

-- adding a sector to the where clause causes ALL 'GB' partitions to be scanned
EXPLAIN ANALYSE SELECT * FROM part.partitioned_table WHERE country = 'GB' and sector in ('BETA', 'GAMMA');

-- instead adding a country to the where clause causes ALL 'GB' and 'FR' partitions to be scanned!
EXPLAIN ANALYSE SELECT * FROM part.partitioned_table WHERE country in ('GB', 'FR') and sector = 'BETA';

请注意，即使我使用 'OR' 逻辑而不是 'IN'，也会扫描相同的分区。如果需要，我可以添加 explain analyse 输出。

那么首先，为什么仅仅因为指定了一个以上的扇区就扫描所有 'GB' 个分区？

其次，也许更奇怪的是，如果我坚持使用单个扇区（在我的示例中为 'BETA'）并添加第二个国家，而不是扫描所有 'BETA' 分区，它现在扫描每个指定国家/地区的所有分区。

显然，现实世界 table 是数百个国家和部门。我们有查询跨多个国家（例如 20 个）的单个扇区的用例，并最终扫描数百个分区（这 20 个国家/地区的所有扇区分区），当只需要扫描 20 个分区时，这是“显式的” " 在查询中。

我们是否需要创建另一个主要按扇区分区的 table 版本来解决这个问题，还是我们在这里遗漏了什么？

Answer 1

子分区的使用使查询规划器的行为更加明智。由于这也允许我们 partition by list，因此无需人为地将分区键中的整数列包含在范围内：

CREATE TABLE part.sub_partitioned_table
(
    country character varying NOT NULL,
    sector character varying NOT NULL,
    a_value integer NOT NULL,
    other_value integer,
    CONSTRAINT sub_partitioned_table_pkey PRIMARY KEY (country, sector, a_value)
) PARTITION BY LIST (country);

CREATE TABLE part.sub_partitioned_table_de PARTITION OF part.sub_partitioned_table
    FOR VALUES IN ('DE')
    PARTITION BY LIST (sector);
CREATE TABLE part.sub_partitioned_table_gb PARTITION OF part.sub_partitioned_table
    FOR VALUES IN ('GB')
    PARTITION BY LIST (sector);
CREATE TABLE part.sub_partitioned_table_fr PARTITION OF part.sub_partitioned_table
    FOR VALUES IN ('FR')
    PARTITION BY LIST (sector);

CREATE TABLE part.sub_partitioned_table_gb_alpha PARTITION OF part.sub_partitioned_table_gb
    FOR VALUES IN ('ALPHA');
CREATE TABLE part.sub_partitioned_table_gb_beta PARTITION OF part.sub_partitioned_table_gb
     FOR VALUES IN ('BETA');
CREATE TABLE part.sub_partitioned_table_gb_gamma PARTITION OF part.sub_partitioned_table_gb
    FOR VALUES IN ('GAMMA');
CREATE TABLE part.sub_partitioned_table_fr_alpha PARTITION OF part.sub_partitioned_table_fr
    FOR VALUES IN ('ALPHA');
CREATE TABLE part.sub_partitioned_table_fr_beta PARTITION OF part.sub_partitioned_table_fr
     FOR VALUES IN ('BETA');
CREATE TABLE part.sub_partitioned_table_fr_gamma PARTITION OF part.sub_partitioned_table_fr
    FOR VALUES IN ('GAMMA');    
CREATE TABLE part.sub_partitioned_table_de_alpha PARTITION OF part.sub_partitioned_table_de
    FOR VALUES IN ('ALPHA');
CREATE TABLE part.sub_partitioned_table_de_beta PARTITION OF part.sub_partitioned_table_de
     FOR VALUES IN ('BETA');
CREATE TABLE part.sub_partitioned_table_de_gamma PARTITION OF part.sub_partitioned_table_de
    FOR VALUES IN ('GAMMA');

INSERT INTO part.sub_partitioned_table(country, sector, a_value, other_value) VALUES
    ('GB', 'ALPHA', 10, 1000),
    ('GB', 'BETA', 10, 1000),
    ('GB', 'GAMMA', 10, 1000),
    ('FR', 'ALPHA', 10, 1000),
    ('FR', 'BETA', 10, 1000),
    ('FR', 'GAMMA', 10, 1000),
    ('DE', 'ALPHA', 10, 1000),
    ('DE', 'BETA', 10, 1000),
    ('DE', 'GAMMA', 10, 1000);
    
    
EXPLAIN ANALYSE SELECT * FROM part.sub_partitioned_table WHERE country = 'GB' and sector = 'BETA';

EXPLAIN ANALYSE SELECT * FROM part.sub_partitioned_table WHERE country = 'GB' and sector in ('ALPHA', 'BETA');
EXPLAIN ANALYSE SELECT * FROM part.sub_partitioned_table WHERE country = 'GB' and (sector = 'ALPHA' or sector = 'BETA');

EXPLAIN ANALYSE SELECT * FROM part.sub_partitioned_table WHERE country in ('GB', 'FR') and sector = 'BETA';
EXPLAIN ANALYSE SELECT * FROM part.sub_partitioned_table WHERE (country = 'GB' or country = 'FR') and sector = 'BETA';

EXPLAIN ANALYSE SELECT * FROM part.sub_partitioned_table WHERE
((sector = 'BETA' AND country = 'GB') OR
 (sector = 'BETA' AND country = 'FR'))
 
EXPLAIN ANALYSE SELECT * FROM part.sub_partitioned_table WHERE
((sector = 'BETA' AND country = 'GB') OR
 (sector = 'ALPHA' AND country = 'GB'))

在所有情况下，查询计划都显示只扫描相关（子）分区。

为什么 Postgres 多列分区修剪不比这更聪明？

Why isn't Postgres multicolumn partition pruning smarter than this?

postgresql

partitioning

database-partitioning