我需要在java连接hive,所以要启动hiveserver2，先贴一下hive-site.xml
<property> <name>hive.server2.thrift.bind.host</name> <value>10.10.22.133</value> <description>Bind host on which to run the HiveServer2 Thrift service.</description> </property> <property> <name>hive.metastore.uris</name> <value>thrift://master:9083</value> </property> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://master:3306/hivemetastore</value> <description/> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> <description/> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> <description/> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>123456</value> <description/> </property> <property> <name>hive.execution.engine</name> <value>spark</value> </property> <property> <name>spark.home</name> <value>/opt/hadoop/spark-2.3.0-bin-hadoop2.7</value> </property> <property> <name>spark.master</name> <value>yarn-cluster</value> </property>
1.发现问题启动不了端口10000
先启动metastore
nohup hive --service metastore &
再启动 hiveserver2
nohup hive --service hiveserver2 &
ps -ef|grep hive
发现两个都有进程
netstat -tunlp
只有9083 没有10000端口
2.开启hive日志分析问题
cd /opt/hadoop/apache-hive-3.0.0-bin/conf/ cp hive-log4j2.properties.template hive-log4j2.properties cat hive-log4j2.properties
发现内容：property.hive.log.dir = ${sys:java.io.tmpdir}/${sys:user.name}
那我的hive日志就在/tmp/root下面,监控日志
tail -f /tmp/root/hive.log
新开一个ssh窗口
kill -9 hiveserver2进程 hiveserver2
3.get_current_notificationEventId错误日志输出
2019-01-10T15:52:57,275 WARN [main] metastore.RetryingMetaStoreClient: MetaStoreClient lost connection. Attempting to reconnect (1 of 1) after 1s. getCurrentNotificationEventId org.apache.thrift.TApplicationException: Internal error processing get_current_notificationEventId at org.apache.thrift.TApplicationException.read(TApplicationException.java:111) ~[hive-exec-3.0.0.jar:3.0.0] at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79) ~[hive-exec-3.0.0.jar:3.0.0] at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_current_notificationEventId(ThriftHiveMetastore.java:5541) ~[hive-exec-3.0.0.jar:3.0.0] at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_current_notificationEventId(ThriftHiveMetastore.java:5529) ~[hive-exec-3.0.0.jar:3.0.0] at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getCurrentNotificationEventId(HiveMetaStoreClient.java:2713) ~[hive-exec-3.0.0.jar:3.0.0] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_77] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_77] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_77] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_77] at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212) ~[hive-exec-3.0.0.jar:3.0.0] at com.sun.proxy.$Proxy34.getCurrentNotificationEventId(Unknown Source) ~[?:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_77] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_77] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_77] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_77] at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2763) ~[hive-exec-3.0.0.jar:3.0.0] at com.sun.proxy.$Proxy34.getCurrentNotificationEventId(Unknown Source) ~[?:?] 。。。。。。 at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_77] at org.apache.hadoop.util.RunJar.run(RunJar.java:226) [hadoop-common-2.7.7.jar:?] at org.apache.hadoop.util.RunJar.main(RunJar.java:141) [hadoop-common-2.7.7.jar:?] Caused by: java.io.IOException: org.apache.thrift.TApplicationException: Internal error processing get_current_notificationEventId at org.apache.hadoop.hive.metastore.messaging.EventUtils$MSClientNotificationFetcher.getCurrentNotificationEventId(EventUtils.java:75) ~[hive-exec-3.0.0.jar:3.0.0] at org.apache.hadoop.hive.ql.metadata.events.NotificationEventPoll.<init>(NotificationEventPoll.java:103) ~[hive-exec-3.0.0.jar:3.0.0] at org.apache.hadoop.hive.ql.metadata.events.NotificationEventPoll.initialize(NotificationEventPoll.java:59) ~[hive-exec-3.0.0.jar:3.0.0] at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:267) ~[hive-service-3.0.0.jar:3.0.0] ... 10 more Caused by: org.apache.thrift.TApplicationException: Internal error processing get_current_notificationEventId
解决方法：
修改 hive-site.xml添加(我用的这个方式 问题解决)
<property> <name>hive.metastore.event.db.notification.api.auth</name> <value>false</value> </property>
或者：
hadoop core-site.xml添加
hadoop.proxyuser.hive.hosts=HS2_HOST hadoop.proxyuser.hive.groups=*
继续启动并监控日志
4.接着报错SessionNotRunning
2019-01-10T16:00:59,680 WARN [main] server.HiveServer2: Error starting HiveServer2 on attempt 1, will retry in 60000ms java.lang.NoClassDefFoundError: org/apache/tez/dag/api/SessionNotRunning at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession$AbstractTriggerValidator.startTriggerValidator(TezSessionPoolSession.java:74) ~[hive-exec-3.0.0.jar:3.0.0] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.initTriggers(TezSessionPoolManager.java:207) ~[hive-exec-3.0.0.jar:3.0.0] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.startPool(TezSessionPoolManager.java:114) ~[hive-exec-3.0.0.jar:3.0.0] at org.apache.hive.service.server.HiveServer2.initAndStartTezSessionPoolManager(HiveServer2.java:831) ~[hive-service-3.0.0.jar:3.0.0] at org.apache.hive.service.server.HiveServer2.startOrReconnectTezSessions(HiveServer2.java:815) ~[hive-service-3.0.0.jar:3.0.0] at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:739) ~[hive-service-3.0.0.jar:3.0.0] at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1014) [hive-service-3.0.0.jar:3.0.0] at org.apache.hive.service.server.HiveServer2.access$1800(HiveServer2.java:134) [hive-service-3.0.0.jar:3.0.0] at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1282) [hive-service-3.0.0.jar:3.0.0] at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1126) [hive-service-3.0.0.jar:3.0.0] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_77] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_77] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_77] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_77] at org.apache.hadoop.util.RunJar.run(RunJar.java:226) [hadoop-common-2.7.7.jar:?] at org.apache.hadoop.util.RunJar.main(RunJar.java:141) [hadoop-common-2.7.7.jar:?] Caused by: java.lang.ClassNotFoundException: org.apache.tez.dag.api.SessionNotRunning at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_77] at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_77] at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[?:1.8.0_77] at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_77] ... 16 more
额 我配置用的 spark engine,找tez类干什么，为什么
带着这个为什么，我仔细看日志发现了
2019-01-10T16:45:02,837 INFO [main] server.HiveServer2: HS2 interactive HA not enabled. Starting tez sessions.. 2019-01-10T16:45:02,837 INFO [main] server.HiveServer2: Starting/Reconnecting tez sessions..
就是这个HS2 interactive HA not enabled，然后转向了启动 tez
解决方法：
hive-site.xml追加配置，默认配置是false：
<property> <name>hive.server2.active.passive.ha.enable</name> <value>true</value> </property>
再次启动
nohup hive --service hiveserver2 &
没有报错了
netstat -tunlp tcp 0 0 0.0.0.0:10000 0.0.0.0:* LISTEN 4112/java tcp 0 0 0.0.0.0:10002 0.0.0.0:* LISTEN 4112/java
查看端口已启动，应该是解决了
5.beeline客户端连接测试
先访问一下http://master:10002/
beeline客户端连接hive
./beeline -u jdbc:hive2://master:10000
报错
Error: Could not open client transport with JDBC Uri: jdbc:hive2://master:10000: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous (state=08S01,code=0) Beeline version 3.0.0 by Apache Hive
就是用户验证不通过呗
修改hadoop core-site.xml 追加
<property> <name>hadoop.proxyuser.root.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.root.groups</name> <value>*</value> </property>
重启hadoop,ok！
再连
[root@master bin]# ./beeline -u jdbc:hive2://master:10000 Connecting to jdbc:hive2://master:10000 Connected to: Apache Hive (version 3.0.0) Driver: Hive JDBC (version 3.0.0) Transaction isolation: TRANSACTION_REPEATABLE_READ Beeline version 3.0.0 by Apache Hive 0: jdbc:hive2://master:10000> show databases; INFO : Compiling command(queryId=root_20190110173600_9c92ad1f-b0ca-43a6-ae54-d52decee31af): show databases INFO : Concurrency mode is disabled, not creating a lock manager INFO : Semantic Analysis Completed INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null) INFO : Completed compiling command(queryId=root_20190110173600_9c92ad1f-b0ca-43a6-ae54-d52decee31af); Time taken: 1.392 seconds INFO : Concurrency mode is disabled, not creating a lock manager INFO : Executing command(queryId=root_20190110173600_9c92ad1f-b0ca-43a6-ae54-d52decee31af): show databases INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=root_20190110173600_9c92ad1f-b0ca-43a6-ae54-d52decee31af); Time taken: 0.053 seconds INFO : OK INFO : Concurrency mode is disabled, not creating a lock manager +----------------+ | database_name | +----------------+ | datacenter | | default | +----------------+ 2 rows selected (2.019 seconds) 0: jdbc:hive2://master:10000>
遇到问题还是要慢慢仔细解决，静下来 静下来！！！
