Java连接Phoenix(HBase) 超时的问题

phoenix | 2019-09-19 09:28:06

前言背景:

用到HBase的背景也是因为数据量到达了一定的量级,传统的关系型数据库存储处理有些乏力,故转为HBase这种扩展性更强、处理大数据更强的存储方式。

连接示例代码

try {
        Connection con =
                DriverManager.getConnection("jdbc:phoenix:hb-proxy-pub1,hb-proxy-pub2,hb-proxy-pub3", props);
        Statement stmt = con.createStatement();
        stmt.execute("drop table if exists test");
        stmt.execute("create table test (mykey integer not null primary key, mycolumn varchar)");
        stmt.execute("create index test_idx on test(mycolumn)");
        stmt.executeUpdate("upsert into test values (1,'World!')");
        stmt.executeUpdate("upsert into test values (2,'Hello')");
        stmt.executeUpdate("upsert into test values (3,'World!')");
        con.commit();
        PreparedStatement statement = con.prepareStatement("select mykey from test where mycolumn='Hello'");
        ResultSet rset = statement.executeQuery();
        while (rset.next()) {
            System.out.println(rset.getInt(1));
        }

        rset.close();
        con.close();
    } catch (SQLException e) {
        e.printStackTrace();
    }

连接超时

当检索大量数据的时候,可能会在等待一段时间后报错:

java.util.concurrent.ExecutionException: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:

Caused by: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:

Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions:

Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=60208:

Caused by: java.io.IOException: Call to xxx failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=76, waitTime=60049, operationTimeout=59999 expired.
    at org.apache.hadoop.hbase.ipc.AbstractRpcClient.wrapException(AbstractRpcClient.java:284)
    at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1238)
    at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223)
    at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:328)
    at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32831)
    at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:215)
    at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:63)
    at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
    at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:364)
    at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:338)
    at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:136)
    ... 4 more
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=76, waitTime=60049, operationTimeout=59999 expired.
    at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
    at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1212)
    ... 13 more

[INFO ][phoenix-1-thread-0] 2018-05-29 20:12:47.839 - BaseResultIterators.close(1096) | Failed to execute task during cancel
java.util.concurrent.ExecutionException: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Tue May 29 20:12:28 AWST 2018, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=60202:

错误显示是超过了默认的60秒超时限制,网上有很多解决方案说修改phoenix的超时设置,然而本例并未安装phoenix客户端!
那么如何在Java通过phoenix连接HBase时设置超时时间呢?
查了很多最终找到一个回复指引了正确方向:

From: Nick Dimiduk 2015-09-17, 22:33
I really recommend managing hbase-site.xml outside of your uberjar. You
should already be doing this anyway with other resources, such as
log4j.properties.

If you are intent on setting them programmatically, you’ll need to pass
them into the PhoenixConnection’s HBaseConfiguration object. I believe this
can be done by using the factory method that accepts a Properties instance,
i.e., DriverManager#getConnection(String, Properties).

感谢原作者,经测试成功。

解决方案

在取得Connection时传入Properties对象,并配置相关属性。
phoenix.query.timeoutMs
hbase.rpc.timeout
hbase.client.scanner.timeout.period

Properties props = new Properties();
        props.setProperty("phoenix.query.timeoutMs", "1200000");
        props.setProperty("hbase.rpc.timeout", "1200000");
        props.setProperty("hbase.client.scanner.timeout.period", "1200000");
        try {
            Connection con =
                    DriverManager.getConnection("jdbc:phoenix:xxx", props);

 

登录后即可回复 登录 | 注册
    
关注编程学问公众号