无法将 Impala-Kudu 连接到 Apache Kudu(没有 Cloudera Manager):获取 TTransportException 错误
Cannot connect Impala-Kudu to Apache Kudu (without Cloudera Manager): Get TTransportException Error
我已经根据官方 kudu 文档(参见 http://kudu.apache.org/docs/installation.html ). The setup has one node running master and tablet server and another node running the tablet server only. I am having issues installing impala-kudu without Cloudera Manager on the node running kudu master. I have followed CDH installation instructions on this (see http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_cdh5_install.html )页面在 Ubuntu (Trusty) 上成功安装了 kudu,直到第 3 步。我避免使用 YARN 和 MRv1 安装 CDH,因为我不不需要 运行 任何 mapreduce 作业,也不会使用 hadoop。 Impala-kudu 和 impala-kudu-shell 安装没有错误。当我启动 impala-shell 它 returns:
Starting Impala Shell without Kerberos authentication
Error connecting: TTransportException, Could not connect to kudu_test:21000
***********************************************************************************
Welcome to the Impala shell. Copyright (c) 2015 Cloudera, Inc. All rights reserved.
(Impala Shell v2.7.0-cdh5-IMPALA_KUDU-cdh5 (48f1ad3) built on Thu Aug 18 12:15:44 PDT 2016)Want to know what version of Impala you're connected to? Run the VERSION command to
find out!
***********************************************************************************
[Not connected] >
我尝试使用CONNECT 选项连接到kudu-master 节点但没有成功。 imapala-kudu 和 kudu 在同一台机器上 运行ning。是否需要更改其他配置设置,或者 hadoop 和 YARN 是否是使 impala-kudu 工作的严格要求?
在 运行ning ps -ef | grep -i impalad
之后,我可以确认 impala 守护程序不是 运行ning。导航到位于 ~/var/log/impala 的 impala 日志后,我发现了一些错误和警告文件。这是 impalad.ERROR 的输出:
Log file created at: 2016/09/13 13:26:24
Running on machine: kudu_test
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0913 13:26:24.084389 3021 logging.cc:118] stderr will be logged to this file.
E0913 13:26:25.406966 3021 impala-server.cc:249] Currently configured default filesystem: LocalFileSystem. fs.defaultFS (file:///) is not supported.ERROR: block location tracking is not properly enabled because
- dfs.datanode.hdfs-blocks-metadata.enabled is not enabled.
- dfs.client.file-block-storage-locations.timeout.millis is too low. It should be at least 10 seconds.
E0913 13:26:25.406990 3021 impala-server.cc:252] Aborting Impala Server startup due to improper configuration. Impalad exiting.
也许我需要重新访问 HDFS 和 Hive Metastore 以确保我正确配置了这些服务?
根据日志,impalad 退出是因为默认文件系统配置为不支持的 LocalFileSystem。您必须设置一个分布式文件系统,例如默认的HDFS。
尽管 Kudu 是一个独立的存储系统并且不依赖于 HDFS,Impala 即使与 Kudu 一起使用时似乎仍然需要非本地默认 FS。 Impala_Kudu documentation 明确列出了以下要求:
Before installing Impala_Kudu, you must have already installed and configured services for HDFS (though it is not used by Kudu), the Hive Metastore (where Impala stores its metadata), and Kudu.
我什至可以想象,除了让 Impala 开心之外,可能真的不需要 HDFS,但这只是我的猜测。更新:发现 IMPALA-1850 这证实了我的怀疑 Impala 不再需要 HDFS,但这不仅仅是必须删除的单个检查。
我已经根据官方 kudu 文档(参见 http://kudu.apache.org/docs/installation.html ). The setup has one node running master and tablet server and another node running the tablet server only. I am having issues installing impala-kudu without Cloudera Manager on the node running kudu master. I have followed CDH installation instructions on this (see http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_cdh5_install.html )页面在 Ubuntu (Trusty) 上成功安装了 kudu,直到第 3 步。我避免使用 YARN 和 MRv1 安装 CDH,因为我不不需要 运行 任何 mapreduce 作业,也不会使用 hadoop。 Impala-kudu 和 impala-kudu-shell 安装没有错误。当我启动 impala-shell 它 returns:
Starting Impala Shell without Kerberos authentication
Error connecting: TTransportException, Could not connect to kudu_test:21000
***********************************************************************************
Welcome to the Impala shell. Copyright (c) 2015 Cloudera, Inc. All rights reserved.
(Impala Shell v2.7.0-cdh5-IMPALA_KUDU-cdh5 (48f1ad3) built on Thu Aug 18 12:15:44 PDT 2016)Want to know what version of Impala you're connected to? Run the VERSION command to
find out!
***********************************************************************************
[Not connected] >
我尝试使用CONNECT 选项连接到kudu-master 节点但没有成功。 imapala-kudu 和 kudu 在同一台机器上 运行ning。是否需要更改其他配置设置,或者 hadoop 和 YARN 是否是使 impala-kudu 工作的严格要求?
在 运行ning ps -ef | grep -i impalad
之后,我可以确认 impala 守护程序不是 运行ning。导航到位于 ~/var/log/impala 的 impala 日志后,我发现了一些错误和警告文件。这是 impalad.ERROR 的输出:
Log file created at: 2016/09/13 13:26:24
Running on machine: kudu_test
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0913 13:26:24.084389 3021 logging.cc:118] stderr will be logged to this file.
E0913 13:26:25.406966 3021 impala-server.cc:249] Currently configured default filesystem: LocalFileSystem. fs.defaultFS (file:///) is not supported.ERROR: block location tracking is not properly enabled because
- dfs.datanode.hdfs-blocks-metadata.enabled is not enabled.
- dfs.client.file-block-storage-locations.timeout.millis is too low. It should be at least 10 seconds.
E0913 13:26:25.406990 3021 impala-server.cc:252] Aborting Impala Server startup due to improper configuration. Impalad exiting.
也许我需要重新访问 HDFS 和 Hive Metastore 以确保我正确配置了这些服务?
根据日志,impalad 退出是因为默认文件系统配置为不支持的 LocalFileSystem。您必须设置一个分布式文件系统,例如默认的HDFS。
尽管 Kudu 是一个独立的存储系统并且不依赖于 HDFS,Impala 即使与 Kudu 一起使用时似乎仍然需要非本地默认 FS。 Impala_Kudu documentation 明确列出了以下要求:
Before installing Impala_Kudu, you must have already installed and configured services for HDFS (though it is not used by Kudu), the Hive Metastore (where Impala stores its metadata), and Kudu.
我什至可以想象,除了让 Impala 开心之外,可能真的不需要 HDFS,但这只是我的猜测。更新:发现 IMPALA-1850 这证实了我的怀疑 Impala 不再需要 HDFS,但这不仅仅是必须删除的单个检查。