在配置单元中使用 Buckets 创建 Avro table

Question

我创建了一个带桶的 avro table，但我遇到以下错误：

Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Bucket columns uniqueid is not part of the table columns ([]

CREATE TABLE s.TEST_OD_V(
UniqueId int,
dtCd string,
SysSK int,
Ind string)
PARTITIONED BY (vcd STRING)
CLUSTERED BY (UniqueId) INTO 500 BUCKETS
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT                'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
TBLPROPERTIES ('avro.schema.url'='s3a:/bucket/schema/pr_v.avsc');

我正在使用配置单元 1.1。请帮助我..

Answer 1

试试这个（可从 Hive 0.14 获得）：

CREATE TABLE s.TEST_OD_V( UniqueId int, dtCd string, SysSK int, Ind string) PARTITIONED BY (vcd STRING) CLUSTERED BY (UniqueId) INTO 500 BUCKETS STORED AS AVRO;

在配置单元中使用 Buckets 创建 Avro table

Creating Avro table with Buckets in hive

hadoop

hive

amazon-s3

avro