Titan DB - Hbase兼容性问题

问题描述 投票:0回答:2

尝试使用Titan DB 1.0.0将数据加载到HBase 1.2.0中但我得到以下错误:我知道HBase版本http://s3.thinkaurelius.com/docs/titan/1.0.0/version-compat.html存在兼容性问题,所以我添加了这个:conf.setProperty("storage.hbase.compat-class", "com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat1_0")以下是我的spark-submit工作:

spark-submit --jars javassist-3.12.1.GA.jar,titan-hadoop-core-1.0.0.jar,titan-hbase-1.0.0.jar,titan-hadoop-2-1.0.0.jar,blueprints-core-2.6.0.jar,gremlin-core-3.0.1-incubating.jar,guava-16.0.jar,high-scale-lib-1.1.2.jar,titan-core-1.0.0.jar,titan-factory-0.12.1.jar,/usr/iop/4.2.0.0/hbase/lib/hbase-client-1.2.0-IBM-7.jar,/usr/iop/4.2.0.0/hbase/lib/hbase-hadoop-compat-1.2.0-IBM-7.jar,/usr/iop/4.2.0.0/hbase/lib/htrace-core-3.1.0-incubating.jar,/usr/iop/4.2.0.0/hbase/lib/hbase-common-1.2.0-IBM-7.jar,/usr/iop/4.2.0.0/hbase/lib/hbase-hadoop2-compat-1.2.0-IBM-7.jar,/usr/iop/4.2.0.0/hbase/lib/hbase-protocol.jar,/usr/iop/4.2.0.0/hbase/lib/hbase-server-1.2.0-IBM-7.jar,/usr/iop/4.2.0.0/hbase/lib/metrics-core-2.2.0.jar,/usr/iop/4.2.0.0/hbase/lib/hbase-annotations-1.2.0-IBM-7.jar --class com.souvik.titandb.testTitan cscdentalstreaming-0.0.1-SNAPSHOT.jar --driver-memory 20G --executor-memory 10G --num-executors 32

Exception in thread "main" java.lang.NoClassDefFoundError: org/reflections/Configuration
    at com.thinkaurelius.titan.diskstorage.configuration.ConfigNamespace.getChild(ConfigNamespace.java:68)
    at com.thinkaurelius.titan.diskstorage.configuration.ConfigElement.parse(ConfigElement.java:180)
    at com.thinkaurelius.titan.diskstorage.configuration.BasicConfiguration.getAll(BasicConfiguration.java:80)
    at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.<init>(GraphDatabaseConfiguration.java:1327)
    at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:94)
    at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:74)
    at com.metlife.titandb.testTitan$.getTitanConnection(testTitan.scala:330)
    at com.metlife.titandb.testTitan$.createSchema(testTitan.scala:206)
    at com.metlife.titandb.testTitan$.main(testTitan.scala:340)
    at com.metlife.titandb.testTitan.main(testTitan.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
    Caused by: java.lang.ClassNotFoundException: org.reflections.Configuration
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
scala apache-spark hbase titan
2个回答
1
投票

Titan不再维护,因此它不支持更新版本的存储后端。

JanusGraph,泰坦的一个分支,开箱即用compatibility with HBase 1.2,所以你可以尝试使用它。

GitHub主持JanusGraph releases

© www.soinside.com 2019 - 2024. All rights reserved.