使用 MapDriver 测试 Hbase 连接

Question

我正在使用 MapDriver 来测试 Mapper 程序。在 Mapper - setup() 方法中，我正在实例化一个 Hbase table，它正在尝试连接到 zookeeper。
在我的测试中 class 我无法连接到 zookeeper，它一直因连接超时异常而失败。

SampleMapper.java
@Override
public void setup(Context context) throws IOException, InterruptedException {
    Configuration config = context.getConfiguration();
    table = new HTable(config, config.get("tableName"));
    Scan tableScan = new Scan();
    tableScan.addColumn(COLUMN_FAMILY_STUDENT, COLUMN_ID);
    tableScan.setCaching(NUMBER_OF_ROWS_TO_CACHE);
    tableScan.setCacheBlocks(false);
    super.setup(context);
}

sampleMapperTest.java
@Before
public void setUp() {
    mapDriver = MapDriver.newMapDriver(new SampleMapper());
    Configuration conf = mapDriver.getConfiguration();
    conf.set("tableName", "testTable");
    conf.setStrings("io.serializations",
            conf.get("io.serializations"), MutationSerialization.class.getName(), ResultSerialization.class.getName());

}


 [org.apache.zookeeper.ZooKeeper] Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection-0x625e134e, quorum=localhost:2181, baseZNode=/hbase
  main-SendThread(127.0.0.1:2181) INFO  [org.apache.zookeeper.ClientCnxn] Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
  main-SendThread(127.0.0.1:2181) WARN  [org.apache.zookeeper.ClientCnxn] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
    java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
        04 Mar 2018 20:31 47,648 main-SendThread(0:0:0:0:0:0:0:1:2181) INFO  [org.apache.zookeeper.ClientCnxn] Opening socket connection to server 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
        04 Mar 2018 20:31 47,702 main WARN  [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
         04 Mar 2018 20:31 47,702 main INFO  [org.apache.hadoop.hbase.util.RetryCounter] Sleeping 1000ms before retry #0...
        04 Mar 2018 20:31 48,651 main-SendThread(0:0:0:0:0:0:0:1:2181) WARN  [org.apache.zookeeper.ClientCnxn] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
        java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
            04 Mar 2018 20:31 48,752 main WARN  [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
       04 Mar 2018 20:31 48,752 main INFO  [org.apache.hadoop.hbase.util.RetryCounter] Sleeping 2000ms before retry #1...
       04 Mar 2018 20:31 49,753 main-SendThread(127.0.0.1:2181) INFO  [org.apache.zookeeper.ClientCnxn] Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
       04 Mar 2018 20:31 50,755 main-SendThread(127.0.0.1:2181) WARN  [org.apache.zookeeper.ClientCnxn] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
    java.net.ConnectException: Connection refused: no further information

我不知道如何在这里嘲笑动物园管理员。我怎样才能连接到动物园管理员？

有人可以帮忙吗？

Answer 1

main-SendThread(127.0.0.1:2181) INFO [org.apache.zookeeper.ClientCnxn] Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) main-SendThread(127.0.0.1:2181) WARN [org.apache.zookeeper.ClientCnxn] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information

您的错误表明您正在连接到本地主机 (127.0.0.1)。如果您使用的是远程主机，请设置以下内容：

config.set("hbase.zookeeper.quorum", "remote ip");
config.set("hbase.zookeeper.property.clientPort", "2181"); //your port
config.set("hbase.master", "hbase_master_ip:16000"); //your port

使用 MapDriver 测试 Hbase 连接

test Hbase connection using MapDriver

hadoop

junit

hbase

apache-zookeeper

hadoop2