配置单元 0.14：无法在没有存储桶的情况下将数据插入支持的 ACID table
hive 0.14:can not insert data in a ACID supported table with out buckets
在 hive 0.14 中，我有一个 table 支持 ACID 的事务。
create table HiveTest
(EmployeeID Int,FirstName String,Designation String,
Salary Int,Department String)
clustered by (department) into 3 buckets
stored as orc TBLPROPERTIES ('transactional'='true');
OK.i可以插入数据到这个table:
from stagingtbl
insert into table HiveTest
select employeeid,firstname,designation,salary,department;
但是如果我不使用这个 table:
create table HiveTest
(EmployeeID Int,FirstName String,Designation String,
Salary Int,Department String)
stored as orc TBLPROPERTIES ('transactional'='true');
插入 data:tables 未分桶时我会收到错误消息。
所以我们必须创建一个 table 带桶的 ACID 事务支持来插入数据？
我们可以使用任何其他方法在支持的 ACID 中插入数据吗 table?
如 Hive Transactions 中的限制部分所述：
- BEGIN, COMMIT, and ROLLBACK are not yet supported. All language operations are auto-commit. The plan is to support these in a future release.
- Only ORC file format is supported in this first release. The feature has been built such that transactions can be used by any storage format that can determine how updates or deletes apply to base records (basically, that has an explicit or implicit row id), but so far the integration work has only been done for ORC.
- By default transactions are configured to be off. See the Configuration section below for a discussion of which values need to be set to configure it.
- Tables must be bucketed to make use of these features. Tables in the same system not using transactions and ACID do not need to be bucketed.
- At this time only snapshot level isolation is supported. When a given query starts it will be provided with a consistent snapshot of the data. There is no support for dirty read, read committed, repeatable read, or serializable. With the introduction of BEGIN the intention is to support snapshot isolation for the duration of transaction rather than just a single query. Other isolation levels may be added depending on user requests.
- The existing ZooKeeper and in-memory lock managers are not compatible with transactions. There is no intention to address this issue. See Basic Design below for a discussion of how locks are stored for transactions.
您使用的是哪个版本的 Hive？ Hive 0.13 支持事务属性，但部分支持 Atomicity 、 Consistency 和 Durabality 但不支持 Isolation。 Hive 0.14 支持但仅在 分桶表 和 Table 上应存储为 ORC 格式 。确保在 HIVE_HOME 中配置 hive-site.xml 以支持并发和动态分区以及其他属性。
在 hive 0.14 中，我有一个 table 支持 ACID 的事务。
create table HiveTest
(EmployeeID Int,FirstName String,Designation String,
Salary Int,Department String)
clustered by (department) into 3 buckets
stored as orc TBLPROPERTIES ('transactional'='true');
OK.i可以插入数据到这个table:
from stagingtbl
insert into table HiveTest
select employeeid,firstname,designation,salary,department;
但是如果我不使用这个 table:
create table HiveTest
(EmployeeID Int,FirstName String,Designation String,
Salary Int,Department String)
stored as orc TBLPROPERTIES ('transactional'='true');
插入 data:tables 未分桶时我会收到错误消息。
所以我们必须创建一个 table 带桶的 ACID 事务支持来插入数据？
我们可以使用任何其他方法在支持的 ACID 中插入数据吗 table?
如 Hive Transactions 中的限制部分所述：
- BEGIN, COMMIT, and ROLLBACK are not yet supported. All language operations are auto-commit. The plan is to support these in a future release.
- Only ORC file format is supported in this first release. The feature has been built such that transactions can be used by any storage format that can determine how updates or deletes apply to base records (basically, that has an explicit or implicit row id), but so far the integration work has only been done for ORC.
- By default transactions are configured to be off. See the Configuration section below for a discussion of which values need to be set to configure it.
- Tables must be bucketed to make use of these features. Tables in the same system not using transactions and ACID do not need to be bucketed.
- At this time only snapshot level isolation is supported. When a given query starts it will be provided with a consistent snapshot of the data. There is no support for dirty read, read committed, repeatable read, or serializable. With the introduction of BEGIN the intention is to support snapshot isolation for the duration of transaction rather than just a single query. Other isolation levels may be added depending on user requests.
- The existing ZooKeeper and in-memory lock managers are not compatible with transactions. There is no intention to address this issue. See Basic Design below for a discussion of how locks are stored for transactions.
您使用的是哪个版本的 Hive？ Hive 0.13 支持事务属性，但部分支持 Atomicity 、 Consistency 和 Durabality 但不支持 Isolation。 Hive 0.14 支持但仅在 分桶表 和 Table 上应存储为 ORC 格式 。确保在 HIVE_HOME 中配置 hive-site.xml 以支持并发和动态分区以及其他属性。