java jdbc通过spark连接hive 异常Required field 'client_protocol' is unset

spark | 2019-09-13 10:02:39

java jdbc通过spark连接hive 异常Required field 'client_protocol' is unset,我是先启动spark thriftserver,然后通过jdbc操作hive,可是出现异常


SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
19:50:09.719 [main] DEBUG org.apache.hive.jdbc.Utils - Resolved authority: 10.10.22.133:10000
19:50:09.983 [main] DEBUG org.apache.thrift.transport.TSaslTransport - opening transport org.apache.thrift.transport.TSaslClientTransport@1b68ddbd
19:50:27.508 [main] DEBUG org.apache.thrift.transport.TSaslClientTransport - Sending mechanism name PLAIN and initial response of length 12
19:50:27.509 [main] DEBUG org.apache.thrift.transport.TSaslTransport - CLIENT: Writing message with status START and payload length 5
19:50:27.509 [main] DEBUG org.apache.thrift.transport.TSaslTransport - CLIENT: Writing message with status COMPLETE and payload length 12
19:50:27.509 [main] DEBUG org.apache.thrift.transport.TSaslTransport - CLIENT: Start message handled
19:50:27.509 [main] DEBUG org.apache.thrift.transport.TSaslTransport - CLIENT: Main negotiation loop complete
19:50:27.517 [main] DEBUG org.apache.thrift.transport.TSaslTransport - CLIENT: SASL Client receiving last message
19:50:27.569 [main] DEBUG org.apache.thrift.transport.TSaslTransport - CLIENT: Received message with status COMPLETE and payload length 0
19:50:27.643 [main] DEBUG org.apache.thrift.transport.TSaslTransport - writing data length: 147
19:50:27.697 [main] DEBUG org.apache.thrift.transport.TSaslTransport - CLIENT: reading data length: 234
19:50:27.701 [main] ERROR org.apache.hive.jdbc.HiveConnection - Error opening session
org.apache.thrift.TApplicationException: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000, use:database=datacenter})
java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://10.10.22.133:10000/datacenter: Could not establish connection to jdbc:hive2://10.10.22.133:10000/datacenter: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000, use:database=datacenter})
at org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:252)
at org.apache.hive.service.rpc.thrift.TCLIService$Client.recv_OpenSession(TCLIService.java:176)
at org.apache.hive.service.rpc.thrift.TCLIService$Client.OpenSession(TCLIService.java:163)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:715)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:228)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at net.itxw.example.HiveTest.run(HiveTest.java:17)
at net.itxw.example.HiveTest.main(HiveTest.java:45)
at net.itxw.example.HiveTest.run(HiveTest.java:17)
19:50:27.701 [main] WARN org.apache.hive.jdbc.HiveConnection - Failed to connect to 10.10.22.133:10000
error
at net.itxw.example.HiveTest.main(HiveTest.java:45)
Caused by: java.sql.SQLException: Could not establish connection to jdbc:hive2://10.10.22.133:10000/datacenter: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000, use:database=datacenter})
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:734)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:228)
... 5 more
Caused by: org.apache.thrift.TApplicationException: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000, use:database=datacenter})
at org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
at org.apache.hive.service.rpc.thrift.TCLIService$Client.recv_OpenSession(TCLIService.java:176)
at org.apache.hive.service.rpc.thrift.TCLIService$Client.OpenSession(TCLIService.java:163)
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:715)
... 6 more


解决方法:

这个异常是hive driver版本不对造成的。

jdbc hive driver的jar我是按hive程序来选择版本的,其实不是的,在spark jars里面看hive-jdbc jar的版本,然后在java程序中选择对应的版本即可。

<dependency>
    <groupId>org.apache.hive</groupId>
    <artifactId>hive-jdbc</artifactId>
    <version>1.2.1</version>
</dependency>




登录后即可回复 登录 | 注册
    
关注编程学问公众号