如何插入配置单元 table,按从临时 table 读取的日期分区?
How to insert into hive table, partitioned by date reading from temp table?
我有一个 Hive temp table,没有任何具有所需数据的分区。我想 select 这个数据并插入到另一个按日期分区的 table 中。我尝试了以下技术但没有成功。
来源table架构
CREATE TABLE cls_staging.cls_billing_address_em_tmp
( col1 string,
col2 string,
col3 string);
目的地table:
CREATE TABLE cls_staging.cls_billing_address_em_tmp
( col1 string,
col2 string,
col3 string)
PARTITIONED BY (
curr_date string)
STORED AS ORC;
插入目标的查询table:
insert overwrite table cls_staging.cls_billing_address_em_tmp partition (record_date) select col1, col2, col3, FROM_UNIXTIME(UNIX_TIMESTAMP()) from myDB.mytbl;
错误
Dynamic partition strict mode requires at least one static partition column
第二
insert overwrite table cls_staging.cls_billing_address_em_tmp partition (record_date = FROM_UNIXTIME(UNIX_TIMESTAMP())) select col1, col2, col3 from myDB.mytbl;
错误:
cannot recognize input near 'FROM_UNIXTIME' '(' 'UNIX_TIMESTAMP'
第一次开启动态分区和非严格模式:
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
insert overwrite table cls_staging.cls_billing_address_em_tmp partition (record_date)
select col1, col2, col3, current_timestamp from myDB.mytbl;
2nd:不要用unix_timestamp()
这个目的,因为它会产生很多不同的时间戳,使用current_timestamp
常量,读这个:
我有一个 Hive temp table,没有任何具有所需数据的分区。我想 select 这个数据并插入到另一个按日期分区的 table 中。我尝试了以下技术但没有成功。
来源table架构
CREATE TABLE cls_staging.cls_billing_address_em_tmp
( col1 string,
col2 string,
col3 string);
目的地table:
CREATE TABLE cls_staging.cls_billing_address_em_tmp
( col1 string,
col2 string,
col3 string)
PARTITIONED BY (
curr_date string)
STORED AS ORC;
插入目标的查询table:
insert overwrite table cls_staging.cls_billing_address_em_tmp partition (record_date) select col1, col2, col3, FROM_UNIXTIME(UNIX_TIMESTAMP()) from myDB.mytbl;
错误
Dynamic partition strict mode requires at least one static partition column
第二
insert overwrite table cls_staging.cls_billing_address_em_tmp partition (record_date = FROM_UNIXTIME(UNIX_TIMESTAMP())) select col1, col2, col3 from myDB.mytbl;
错误:
cannot recognize input near 'FROM_UNIXTIME' '(' 'UNIX_TIMESTAMP'
第一次开启动态分区和非严格模式:
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
insert overwrite table cls_staging.cls_billing_address_em_tmp partition (record_date)
select col1, col2, col3, current_timestamp from myDB.mytbl;
2nd:不要用unix_timestamp()
这个目的,因为它会产生很多不同的时间戳,使用current_timestamp
常量,读这个: