无法在 Hive 中删除数据库
Failing to Drop Database in Hive
我正在尝试在每个 运行 之前初始化配置单元中的数据库。
密码是：
command="hive -e \"drop database if exists some_db cascade; create database some_db\"";
eval $command;
执行失败并出现错误：
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to clean up java.sql.SQLSyntaxErrorException: Table 'hive.COMPLETED_COMPACTIONS' doesn't exist
它继续 运行,
eval $command;
我也无法解释多次调用该命令时，它一直交替失败。
感谢您提供有关 Hive 中发生的情况以及如何使其在第一次尝试中发挥作用的建议，谢谢。
详情：
Hive 2.1 运行ning 在 AWS EMR 5.7 上，在 HIVE 1.0
上没有看到这种行为
错误堆栈已满
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to clean up java.sql.SQLSyntaxErrorException: Table 'hive.COMPLETED_COMPACTIONS' doesn't exist
at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:125)
at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:69)
at org.mariadb.jdbc.MariaDbStatement.executeQueryEpilog(MariaDbStatement.java:259)
at org.mariadb.jdbc.MariaDbStatement.execute(MariaDbStatement.java:287)
at org.mariadb.jdbc.MariaDbStatement.executeUpdate(MariaDbStatement.java:470)
at org.mariadb.jdbc.MariaDbStatement.executeUpdate(MariaDbStatement.java:486)
at com.jolbox.bonecp.StatementHandle.executeUpdate(StatementHandle.java:497)
at org.apache.hadoop.hive.metastore.txn.TxnHandler.cleanupRecords(TxnHandler.java:1721)
at org.apache.hadoop.hive.metastore.AcidEventListener.onDropDatabase(AcidEventListener.java:51)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_database_core(HiveMetaStore.java:1098)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_database(HiveMetaStore.java:1130)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:140)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
at com.sun.proxy.$Proxy19.drop_database(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_database.getResult(ThriftHiveMetastore.java:10518)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_database.getResult(ThriftHiveMetastore.java:10502)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.run(TUGIBasedProcessor.java:110)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.run(TUGIBasedProcessor.java:106)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Table 'hive.COMPLETED_COMPACTIONS' doesn't exist
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:479)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.result(AbstractQueryProtocol.java:400)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:381)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:337)
at org.mariadb.jdbc.MariaDbStatement.execute(MariaDbStatement.java:277)
... 27 more
Blockquote
看起来您的 Hive Metastore 架构需要升级。您可以使用 schematool 命令检查架构版本。
示例输出：
Metastore 连接 URL: jdbc:mysql://XXXXXXXXXXXXXXXXXX:3306/hive?createDatabaseIfNotExist=true
Metastore 连接驱动程序：org.mariadb.jdbc.Driver
Metastore 连接用户：hive
蜂巢分发版本：2.1.0
Metastore 架构版本：2.1.0
如果您的 Metastore 架构和 Hive 分发版本不同，请升级架构。
确保您先有一个备份，以防万一。还请记住，可能会更改架构，从而破坏较旧的 Hive 发行版的功能，因此您可能无法在 Hive 发行版之间共享 matastore。
我正在尝试在每个 运行 之前初始化配置单元中的数据库。
密码是：
command="hive -e \"drop database if exists some_db cascade; create database some_db\"";
eval $command;
执行失败并出现错误：
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to clean up java.sql.SQLSyntaxErrorException: Table 'hive.COMPLETED_COMPACTIONS' doesn't exist
它继续 运行,
eval $command;
我也无法解释多次调用该命令时，它一直交替失败。
感谢您提供有关 Hive 中发生的情况以及如何使其在第一次尝试中发挥作用的建议，谢谢。
详情： Hive 2.1 运行ning 在 AWS EMR 5.7 上，在 HIVE 1.0
上没有看到这种行为错误堆栈已满
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to clean up java.sql.SQLSyntaxErrorException: Table 'hive.COMPLETED_COMPACTIONS' doesn't exist at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:125) at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:69) at org.mariadb.jdbc.MariaDbStatement.executeQueryEpilog(MariaDbStatement.java:259) at org.mariadb.jdbc.MariaDbStatement.execute(MariaDbStatement.java:287) at org.mariadb.jdbc.MariaDbStatement.executeUpdate(MariaDbStatement.java:470) at org.mariadb.jdbc.MariaDbStatement.executeUpdate(MariaDbStatement.java:486) at com.jolbox.bonecp.StatementHandle.executeUpdate(StatementHandle.java:497) at org.apache.hadoop.hive.metastore.txn.TxnHandler.cleanupRecords(TxnHandler.java:1721) at org.apache.hadoop.hive.metastore.AcidEventListener.onDropDatabase(AcidEventListener.java:51) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_database_core(HiveMetaStore.java:1098) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_database(HiveMetaStore.java:1130) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:140) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99) at com.sun.proxy.$Proxy19.drop_database(Unknown Source) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_database.getResult(ThriftHiveMetastore.java:10518) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_database.getResult(ThriftHiveMetastore.java:10502) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.run(TUGIBasedProcessor.java:110) at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.run(TUGIBasedProcessor.java:106) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Table 'hive.COMPLETED_COMPACTIONS' doesn't exist at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:479) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.result(AbstractQueryProtocol.java:400) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:381) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:337) at org.mariadb.jdbc.MariaDbStatement.execute(MariaDbStatement.java:277) ... 27 more Blockquote
看起来您的 Hive Metastore 架构需要升级。您可以使用 schematool 命令检查架构版本。
示例输出：
Metastore 连接 URL: jdbc:mysql://XXXXXXXXXXXXXXXXXX:3306/hive?createDatabaseIfNotExist=true Metastore 连接驱动程序：org.mariadb.jdbc.Driver Metastore 连接用户：hive 蜂巢分发版本：2.1.0 Metastore 架构版本：2.1.0
如果您的 Metastore 架构和 Hive 分发版本不同，请升级架构。
确保您先有一个备份，以防万一。还请记住，可能会更改架构，从而破坏较旧的 Hive 发行版的功能，因此您可能无法在 Hive 发行版之间共享 matastore。