Hive 和 hive server2 未启动

Question

我正在尝试使用 apache_hive_2.3.7 和 apache_hadoop_3.3.6 在本地计算机中设置配置单元。当我尝试运行配置单元时出现以下错误：

[sgopi@localhost v2.3.7]$ hive
/usr/bin/which: no hbase in (/home/sgopi/.local/bin:/home/sgopi/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin:/etc/hive/v2.3.7/bin:/etc/hadoop/v3.3.6/bin)

Logging initialized using configuration in jar:file:/etc/hive/v2.3.7/lib/hive-common-2.3.7.jar!/hive-log4j2.properties Async: true
Exception in thread "main" java.lang.ExceptionInInitializerError
        at org.apache.hadoop.hive.ql.plan.TableDesc.setProperties(TableDesc.java:133)
        at org.apache.hadoop.hive.ql.plan.TableDesc.<init>(TableDesc.java:71)
        at org.apache.hadoop.hive.ql.plan.PlanUtils.getDefaultTableDesc(PlanUtils.java:458)
        at org.apache.hadoop.hive.ql.exec.Utilities.<clinit>(Utilities.java:681)
        at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:709)
        at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:654)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:586)
        at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:553)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:750)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:328)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:241)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private volatile java.lang.String java.net.URI.string accessible: module java.base does not "opens java.net" to unnamed module @73ee04c8
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
        at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
        at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
        at org.apache.hadoop.hive.common.StringInternUtils.<clinit>(StringInternUtils.java:57)
        ... 16 more

尝试使用

hive --service hiveserver2

，但该命令无限期地运行，没有任何错误/警告。

[sgopi@localhost bin]$ hive --service hiveserver2 --hiveconf hive.server2.thrift.port=10000 --hiveconf hive.root.logger=INFO,console
/usr/bin/which: no hbase in (/home/sgopi/.local/bin:/home/sgopi/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin:/etc/hive/v2.3.7/bin:/etc/hadoop/v3.3.6/bin)
2023-08-30 11:19:31: Starting HiveServer2

当我检查正在运行的进程时，hiveserver2 正在使用

-Xmx256m

参数运行。我已将值为

-Xmx256m

的所有参数值更新为

-Xmx1g

。然而该命令仍然使用

-Xmx256m

。是否因为内存较少而需要时间？有没有办法增加到1GB/2GB？

sgopi      15079   14307  3 11:19 pts/2    00:00:11 /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java -Dproc_jar -Djava.net.preferIPv4Stack=true -Dproc_hiveserver2 -Dlog4j.configurationFile=hive-log4j2.properties -Djava.util.logging.config.file=/etc/hive/v2.3.7/conf/parquet-logging.properties -Dyarn.log.dir=/etc/hadoop/v3.3.6/logs -Dyarn.log.file=hadoop.log -Dyarn.home.dir=/etc/hadoop/v3.3.6 -Dyarn.root.logger=INFO,console -Djava.library.path=/etc/hadoop/v3.3.6/lib/native **-Xmx256m** -Dhadoop.log.dir=/etc/hadoop/v3.3.6/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/etc/hadoop/v3.3.6 -Dhadoop.id.str=sgopi -Dhadoop.root.logger=INFO,console -Dhadoop.policy.file=hadoop-policy.xml -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /etc/hive/v2.3.7/lib/hive-service-2.3.7.jar org.apache.hive.service.server.HiveServer2 --hiveconf hive.server2.thrift.port=10000 --hiveconf hive.root.logger=INFO,console

我尝试在 hive、hadoop 目录中搜索

-Xmx256m

值，但没有包含该值的参数。

[sgopi@localhost v3.3.6]$ pwd
/etc/hadoop/v3.3.6
[sgopi@localhost v3.3.6]$ grep -ir Xmx256 *
[sgopi@localhost v3.3.6]$ cd /etc/hive/v2.3.7/
[sgopi@localhost v2.3.7]$ grep -ir Xmx256 *
[sgopi@localhost v2.3.7]$

Hive Metastore 已经运行，使用 mysql 作为 Hive Metastore。

[sgopi@localhost v2.3.7]$ ps -ef|grep hive
sgopi      12083       1  0 10:10 ?        00:00:37 /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java -Dproc_jar -Dproc_metastore -Dlog4j.configurationFile=hive-log4j2.properties -Djava.util.logging.config.file=/etc/hive/v2.3.7/conf/parquet-logging.properties -Dyarn.log.dir=/etc/hadoop/v3.3.6/logs -Dyarn.log.file=hadoop.log -Dyarn.home.dir=/etc/hadoop/v3.3.6 -Dyarn.root.logger=INFO,console -Djava.library.path=/etc/hadoop/v3.3.6/lib/native -Xmx256m -Dhadoop.log.dir=/etc/hadoop/v3.3.6/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/etc/hadoop/v3.3.6 -Dhadoop.id.str=sgopi -Dhadoop.root.logger=INFO,console -Dhadoop.policy.file=hadoop-policy.xml -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /etc/hive/v2.3.7/lib/hive-metastore-2.3.7.jar org.apache.hadoop.hive.metastore.HiveMetaStore
sgopi      15869   14307  0 11:40 pts/2    00:00:00 grep --color=auto hive
[sgopi@localhost v2.3.7]$ netstat -lntp|grep 12083
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp6       0      0 :::9083                 :::*                    LISTEN      12083/java
[sgopi@localhost v2.3.7]$

***编辑1： 将堆大小从 256MB 更新为 2GB，如突出显示的Here，仍然是同样的问题。

Answer 1

您可以增加下面的堆大小kike

export HADOOP_HEAPSIZE=2048  # Set heap size to 2GB or adjust as needed
export HADOOP_CLIENT_OPTS="-Xmx${HADOOP_HEAPSIZE}m $HADOOP_CLIENT_OPTS"