HIVE "show partitions" 命令不显示正确的分区
HIVE "show partitions" command do not show correct partitions
我有一个带动态分区的分区 table,
分区字段是国籍和生日,
当我使用select * from emp_new where nationality='China'
时,我得到以下三个记录,
+---------------+--------------+--------------+------------------+----------------------+--------------------+--+
| emp_new.name | emp_new.sex | emp_new.age | emp_new.job | emp_new.nationality | emp_new.birthdate |
+---------------+--------------+--------------+------------------+----------------------+--------------------+--+
| Tony | M | 34 | IT specialist | China | 198202 |
| Katrina | F | 33 | IT specialist | China | 198408 |
| Cathy | F | 30 | IT specialist | China | 198704 |
但是当我 运行 show partitions emp_new partition(nationality='China')
时,我得到以下结果:
+-------------------------------------+--+
| partition |
+-------------------------------------+--+
| nationality=China/birthdate=198408 |
| nationality=China/birthdate=198202 |
| nationality=China/birthdate=198704 |
| nationality=China/birthdate=197509 |
| nationality=China/birthdate=196704 |
| nationality=China/birthdate=197805 |
| nationality=China/birthdate=198201 |
| nationality=China/birthdate=197701 |
| nationality=China/birthdate=196708 |
+-------------------------------------+--+
实际上,我之前使用静态和动态分区(nationality='China', birthdate)
将数据加载到此 table,然后 运行 对 table 进行分类并使用动态分区重新加载 (nationality, birthdate)
稍后.
我不明白为什么旧的分区还在。
Truncate
删除 table 的数据文件。
它不会从 Metastore 中删除分区定义。
它不会删除文件系统目录。
演示
hive> create table mytable (i int) partitioned by (p int);
OK
hive> insert into mytable partition (p) values (1,10),(2,10),(3,20),(4,30),(5,30),(6,30);
OK
hive> select * from mytable;
OK
mytable.i mytable.p
1 10
2 10
3 20
4 30
5 30
6 30
hive> show partitions mytable;
OK
partition
p=10
p=20
p=30
hive> !tree ../local_db/mytable;
../local_db/mytable
├── p=10
│ └── 000000_0
├── p=20
│ └── 000000_0
└── p=30
└── 000000_0
3 directories, 3 files
hive> truncate table mytable;
OK
hive> select * from mytable;
OK
mytable.i mytable.p
hive> show partitions mytable;
OK
partition
p=10
p=20
p=30
hive> !tree ../local_db/mytable;
../local_db/mytable
├── p=10
├── p=20
└── p=30
3 directories, 0 files
我知道原因,
我需要在截断 table 后删除分区,
谢谢
我有一个带动态分区的分区 table, 分区字段是国籍和生日,
当我使用select * from emp_new where nationality='China'
时,我得到以下三个记录,
+---------------+--------------+--------------+------------------+----------------------+--------------------+--+
| emp_new.name | emp_new.sex | emp_new.age | emp_new.job | emp_new.nationality | emp_new.birthdate |
+---------------+--------------+--------------+------------------+----------------------+--------------------+--+
| Tony | M | 34 | IT specialist | China | 198202 |
| Katrina | F | 33 | IT specialist | China | 198408 |
| Cathy | F | 30 | IT specialist | China | 198704 |
但是当我 运行 show partitions emp_new partition(nationality='China')
时,我得到以下结果:
+-------------------------------------+--+
| partition |
+-------------------------------------+--+
| nationality=China/birthdate=198408 |
| nationality=China/birthdate=198202 |
| nationality=China/birthdate=198704 |
| nationality=China/birthdate=197509 |
| nationality=China/birthdate=196704 |
| nationality=China/birthdate=197805 |
| nationality=China/birthdate=198201 |
| nationality=China/birthdate=197701 |
| nationality=China/birthdate=196708 |
+-------------------------------------+--+
实际上,我之前使用静态和动态分区(nationality='China', birthdate)
将数据加载到此 table,然后 运行 对 table 进行分类并使用动态分区重新加载 (nationality, birthdate)
稍后.
我不明白为什么旧的分区还在。
Truncate
删除 table 的数据文件。
它不会从 Metastore 中删除分区定义。
它不会删除文件系统目录。
演示
hive> create table mytable (i int) partitioned by (p int);
OK
hive> insert into mytable partition (p) values (1,10),(2,10),(3,20),(4,30),(5,30),(6,30);
OK
hive> select * from mytable;
OK
mytable.i mytable.p
1 10
2 10
3 20
4 30
5 30
6 30
hive> show partitions mytable;
OK
partition
p=10
p=20
p=30
hive> !tree ../local_db/mytable;
../local_db/mytable
├── p=10
│ └── 000000_0
├── p=20
│ └── 000000_0
└── p=30
└── 000000_0
3 directories, 3 files
hive> truncate table mytable;
OK
hive> select * from mytable;
OK
mytable.i mytable.p
hive> show partitions mytable;
OK
partition
p=10
p=20
p=30
hive> !tree ../local_db/mytable;
../local_db/mytable
├── p=10
├── p=20
└── p=30
3 directories, 0 files
我知道原因, 我需要在截断 table 后删除分区, 谢谢