ERROR: Too many data partitions

Question

我在 vertica.log 中看到以下错误：

2016-09-01 15:30:54.007 TM Moveout:0x7f9438012440-a00000001212c3 [Txn] <INFO> Begin Txn: a00000001212c3 'Moveout: Tuple Mover'
2016-09-01 15:30:54.007 TM Moveout:0x7f9438012440-a00000001212c3 [TM] <INFO> Tuple Mover: moving out projection rosing_epg_program_events_super
2016-09-01 15:30:54.017 TM Moveout:0x7f9438012440-a00000001212c3 [EE] <INFO> (a00000001212c3) Moveout projection staging.rosing_epg_program_events_super
2016-09-01 15:30:54.017 TM Moveout:0x7f9438012440-a00000001212c3 [EE] <INFO> (a00000001212c3) TM Moveout: moving out data in WOS for proj "staging.rosing_epg_program_events_super" to epoch 3061
2016-09-01 15:30:54.017 TM Moveout:0x7f9438012440-a00000001212c3 [EE] <INFO> (a00000001212c3) Executing the moveout plan
2016-09-01 15:30:54.040 TM Moveout:0x7f9438012440-a00000001212c3 [EE] <INFO> SortManager found maxMerges 7 too small(64 MB Assigned).
2016-09-01 15:30:54.040 TM Moveout:0x7f9438012440-a00000001212c3 [EE] <INFO> After disabling optimization, maxMerges becomes 15.
2016-09-01 15:30:54.069 TM Moveout:0x7f9438012440-a00000001212c3 [Txn] <INFO> Rollback Txn: a00000001212c3 'Moveout: (Table: staging.rosing_epg_program_events) (Projection: staging.rosing_epg_program_events_super)'
2016-09-01 15:30:54.070 TM Moveout:0x7f9438012440 <LOG> @v_statistic_node0001: 00000/3298: Event Posted: Event Code:14 Event Id:261 Event Severity: Warning [4] PostedTimestamp: 2016-09-01 16:30:54.069887 ExpirationTimestamp: 2016-09-01 16:31:09.069887 EventCodeDescription: Timer Service Task Error ProblemDescription: threadShim: Too many data partitions DatabaseName: statistic Hostname: rosing-vertica.elt.stag.local
2016-09-01 15:30:54.070 TM Moveout:0x7f9438012440 <ERROR> @v_statistic_node0001: {threadShim} 54000/5060: Too many data partitions
        HINT:  Verify that the table partitioning expression is correct
        LOCATION:  handlePartitionKey, /scratch_a/release/16125/vbuild/vertica/EE/Operators/DataTarget.cpp:1478
2016-09-01 15:30:54.070 TM Moveout:0x7f9438012440 [Util] <INFO> Task 'TM Moveout' enabled

似乎我选择了错误的分区字段并达到了 WOS 中分区的限制 here。

任务 SELECT do_tm_task('moveout'); 引发以下错误：

 Task: moveout
(Table: staging.rosing_schema_migrations) (Projection: staging.rosing_schema_migrations_super)
...
(Table: staging.rosing_epg_program_events) (Projection: staging.rosing_epg_program_events_super)
On node v_statistic_node0001:
  ERROR 5060:  Too many data partitions

(1 row)

有人知道如何解决这个问题吗？

更新:

我无法从中删除分区 table:

ALTER TABLE rosing_epg_program_events REMOVE PARTITIONING

因为这 SQL 引发了同样的错误：数据分区太多

更新 2

我使用 woot 答案解决了这个问题。非常感谢！

这是我的修复步骤：

创建 rosing_epg_program_events 的副本 table:

CREATE TABLE staging.rosing_epg_program_events2
LIKE staging.rosing_epg_program_events;

删除新分区 table:

ALTER TABLE staging.rosing_epg_program_events2 REMOVE PARTITIONING;

将数据从旧的复制到新的table。似乎旧 table 包含在出现问题之前和之后插入的所有 (!) 数据：
```
INSERT /*+ DIRECT */ INTO staging.rosing_epg_program_events2
SELECT * FROM staging.rosing_epg_program_events;
```

降旧table:

DROP TABLE staging.rosing_epg_program_events;

重命名新的 table:

ALTER TABLE staging.rosing_epg_program_events2 RENAME TO rosing_epg_program_events;

运行任何情况下的搬迁操作。现在它工作正常：
```
SELECT do_tm_task('moveout');
```

检查任何情况下的最后一个良好纪元。现在它显示实际值：

SELECT GET_LAST_GOOD_EPOCH();
SELECT * FROM epochs WHERE epoch_number = 3064; // result of previous command

现在似乎一切正常。

Answer 1

执行 CREATE TABLE AS SELECT 或 CREATE TABLE LIKE INCLUDING PROJECTIONS，删除分区，然后 INSERT /*+ DIRECT */ SELECT 复制数据并删除 table，然后重命名。此外，在创建分区时，尝试将粒度定位在 40 个分区以下的某处。您没有指定，但如果使用时间戳，请使用公式提取更细粒度的值。例如，要按月执行，请执行：

EXTRACT (year FROM mydate) * 100 + EXTRACT (month FROM mydate)

您不必担心在 Vertica 分区中使用公式。它对字段使用 min/max 值，而不是直接匹配分区键。