将 Phoenix 与 Cloudera Hbase 一起使用（从 repo 安装）

Question

我可以让 Phoenix 在独立的 Apache Hbase

上工作

（注意，所有这些都是针对 RHEL6.5 上的 Hbase 1.0.0）

然而，对于 Hbase 的 Cloudera 风格，我从来没有在它抛出异常的情况下让它工作。（甚至尝试将 RHEL7 minimal 作为 en OS）

Hbase 1.0 的 Phoenix 4.4 也发生了同样的事情。

hbase(main):001:0> version
1.0.0-cdh5.4.4, rUnknown, Mon Jul  6 16:59:55 PDT 2015

堆栈跟踪：

    [ec2-user@ip-172-31-60-109 phoenix-4.5.0-HBase-1.0-bin]$ bin/sqlline.py localhost:2181:/hbase
Setting property: [isolation, TRANSACTION_READ_COMMITTED]
issuing: !connect jdbc:phoenix:localhost:2181:/hbase none none org.apache.phoenix.jdbc.PhoenixDriver
Connecting to jdbc:phoenix:localhost:2181:/hbase
15/08/06 03:10:25 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/08/06 03:10:26 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-phoenix.properties,hadoop-metrics2.properties
15/08/06 03:10:27 WARN ipc.CoprocessorRpcChannel: Call failed on IOException
org.apache.hadoop.hbase.DoNotRetryIOException: org.apache.hadoop.hbase.DoNotRetryIOException: SYSTEM.CATALOG: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;
    at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:84)
    at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1269)
    at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:11619)
    at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7054)
    at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1746)
    at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1728)
    at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService.callBlockingMethod(ClientProtos.java:31447)
    at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
    at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
    at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
    at org.apache.hadoop.hbase.ipc.RpcExecutor.run(RpcExecutor.java:107)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;
    at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.buildDeletedTable(MetaDataEndpointImpl.java:966)
    at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.loadTable(MetaDataEndpointImpl.java:1042)
    at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1216)
    ... 10 more

    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
    at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
    at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:313)
    at org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1609)
    at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.call(RegionCoprocessorRpcChannel.java:92)
    at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.call(RegionCoprocessorRpcChannel.java:89)
    at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
    at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95)
    at org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:56)
    at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService$Stub.createTable(MetaDataProtos.java:11799)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl.call(ConnectionQueryServicesImpl.java:1273)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl.call(ConnectionQueryServicesImpl.java:1261)
    at org.apache.hadoop.hbase.client.HTable.call(HTable.java:1737)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException): org.apache.hadoop.hbase.DoNotRetryIOException: SYSTEM.CATALOG: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;
    at ...

Answer 1

Cloudera 并未正式支持 Apache phoenix，它仍在 cloudera Labs 中，因此您无法在 cloudera 存储库中找到任何 Cloudera Pheonix tar.gz 文件，您可以在 Cloudera 存储库中找到 Phoenix 的唯一位置是parcel repository，但是parcel只有通过cloudera manager安装才能使用，cloudera Phoenix的最新可用版本是4.3.0。

如果您想在Cloudera Hadoop 发行版上执行Phoenix 4.4 或4.5 版本，您需要使用CDH 依赖jar 重新构建phoenix 库。你不能简单地使用 apache Phoenix tar.gz

步骤如下。

最近我发现Andrew Purtell 在使Phoenix 兼容CDH 版本方面做了大量工作。下面的 link github 页面也有同样的内容。从下面 github link 下载适当的分支。这样可以节省您的时间。

https://github.com/chiastic-security/phoenix-for-cloudera/branches

从 Apache repository 下载 Apache phoenix 4.5 源代码。（如果您是从上面的 github 页面下载，请跳过此步骤）

使用 CDH 依赖 jar 重建源代码 - 更新 pom.xml 和 2 个源文件如下（我的 CDH 版本是 5.4.2）

[h4ck3r@host1 phoenix]$ diff phoenix-4.5_Updated/phoenix-4.5.0-HBase-1.0-src/pom.xml  phoenix-4.5_Orig/phoenix-4.5.0-HBase-1.0-src/pom.xml
28c28
< <!--    <module>phoenix-pig</module> -->
---
>     <module>phoenix-pig</module>
37a38,41
>       <id>apache release</id>
>       <url>https://repository.apache.org/content/repositories/releases/</url>
>     </repository>
>     <repository>
42,43c46,50
<       <id>cloudera</id>
<       <url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
---
>       <id>apache snapshot</id>
>       <url>https://repository.apache.org/content/repositories/snapshots/</url>
>       <snapshots>
>         <enabled>true</enabled>
>       </snapshots>
45d51
<
54d59
<
77,81c82,83
<     <hbase.version>1.0.0-cdh5.4.2</hbase.version>
<     <hadoop-two.version>2.6.0-cdh5.4.2</hadoop-two.version>
/<     <hadoop.version>2.6.0-cdh5.4.2</hadoop.version>
<     <pig.version>0.12.0</pig.version>
<     <flume.version>1.5.0-cdh5.4.2</flume.version>
---
>     <hbase.version>1.0.1</hbase.version>
>     <hadoop-two.version>2.5.1</hadoop-two.version>
84a87,88
>     <hadoop.version>2.5.1</hadoop.version>
>     <pig.version>0.13.0</pig.version>
97a102
>     <flume.version>1.4.0</flume.version>
449,450c454
<
<   <dependency>
---
>       <dependency>
454c458
<       </dependency>
---
>       </dependency>

[h4ck3r@host1 phoenix]$ diff phoenix-4.5_Updated/phoenix-4.5.0-HBase-1.0-src/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/LocalIndexMerger.java  phoenix-4.5_Orig/phoenix-4.5.0-HBase-1.0-src/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/LocalIndexMerger.java
84c84
<                     rss.getServerName(), metaEntries,1);
---
>                     rss.getServerName(), metaEntries);

[h4ck3r@host1 phoenix]$ diff phoenix-4.5_Updated/phoenix-4.5.0-HBase-1.0-src/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/IndexSplitTransaction.java phoenix-4.5_Orig/phoenix-4.5.0-HBase-1.0-src/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/IndexSplitTransaction.java
291c291
<                 daughterRegions.getSecond().getRegionInfo(), server.getServerName(),1);
---
>                 daughterRegions.getSecond().getRegionInfo(), server.getServerName());
978c978
< }
---
> }
\ No newline at end of file

以上构建将在每个子组件的目标目录下创建新的 Jars。
从 Apache 下载 Apache phoenix 4.5 二进制文件 repository
提取 phoenix-4.5.0-HBase-1.0-bin.tar.gz 并用新的 Jars 替换下面的 Phoenix Jars
- phoenix-4.5.0-HBase-1.0-client.jar
- phoenix-4.5.0-HBase-1.0-server-without-antlr.jar
- phoenix-4.5.0-HBase-1.0-client-minimal.jar
- phoenix-assembly-4.5.0-HBase-1.0-tests.jar
- phoenix-4.5.0-HBase-1.0-client-without-hbase.jar
- phoenix-core-4.5.0-HBase-1.0.jar
- phoenix-4.5.0-HBase-1.0-server.jar
替换hbase lib位置的phoenix-4.5.0-HBase-1.0-server.jar和phoenix-core-4.5.0-HBase-1.0.jar并重启hbase。 (在4.7中只有phoenix-4.7.0-cdh5.X.1-server.jar被复制到hbase lib)
从新更新的目录执行 phoenix 命令。

由于一些依赖性问题，phoenix-pig 未得到处理，这只是一种解决方法。

Answer 2

New Apache Phoenix 4.5.2 Package from Cloudera Labs

将 Phoenix 与 Cloudera Hbase 一起使用（从 repo 安装）

Using Phoenix with Cloudera Hbase (installed from repo)

hadoop

hbase

cloudera

phoenix