如何解决java.lang.ClassNotFoundException：org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer

Question

我正在使用tinkerpop + Janus Graph + Spark

build.gradle

compile group: 'org.apache.tinkerpop', name: 'spark-gremlin', version: '3.1.0-incubating'

以下是我们拥有的一些关键配置

spark.serializer: org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer

在引用包含上述类的jar的日志对应的长条目中加载

{"@timestamp":"2020-02-18T07:24:21.720+00:00","@version":1,"message":"Added JAR /opt/data/janusgraph/applib2/spark-gremlin-827a65ae26.jar at spark://gdp-identity-stage.target.com:38876/jars/spark-gremlin-827a65ae26.jar with timestamp 1582010661720","logger_name":"o.a.s.SparkContext","thread_name":"SparkGraphComputer-boss","level":"INFO","level_value":20000}

但是我的SparkGraphComputer提交的spark作业失败，当我们看到执行程序日志时，就看到了

Caused by: java.lang.ClassNotFoundException: org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer

为什么即使加载了相应的jar，也会出现此异常？

任何人，请对此提出建议。

正如我提到的那样，当我在完整异常下面打开一个工作日志时，在火花执行器中看到此异常

Spark Executor Command: "/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/bin/java" "-cp" "/opt/spark/spark-2.4.0/conf/:/opt/spark/spark-2.4.0/jars/*:/opt/hadoop/hadoop-3_1_1/etc/hadoop/" "-Xmx56320M" "-Dspark.driver.port=43137" "-XX:+UseG1GC" "-XX:+PrintGCDetails" "-XX:+PrintGCTimeStamps" "-Xloggc:/opt/spark/gc.log" "-Dtinkerpop.gremlin.io.kryoShimService=org.apache.tinkerpop.gremlin.hadoop.structure.io.HadoopPoolShimService" "org.apache.spark.executor.CoarseGrainedExecutorBackend" "--driver-url" "spark://[email protected]:43137" "--executor-id" "43392" "--hostname" "192.168.192.10" "--cores" "6" "--app-id" "app-20200220094335-0001" "--worker-url" "spark://[email protected]:36845"
========================================

Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1713)
    at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:64)
    at org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:188)
    at org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:281)
    at org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.spark.util.Utils$.classForName(Utils.scala:238)
    at org.apache.spark.SparkEnv$.instantiateClass$1(SparkEnv.scala:259)
    at org.apache.spark.SparkEnv$.instantiateClassFromConf$1(SparkEnv.scala:280)
    at org.apache.spark.SparkEnv$.create(SparkEnv.scala:283)
    at org.apache.spark.SparkEnv$.createExecutorEnv(SparkEnv.scala:200)
    at org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$run$1.apply$mcV$sp(CoarseGrainedExecutorBackend.scala:221)
    at org.apache.spark.deploy.SparkHadoopUtil$$anon$2.run(SparkHadoopUtil.scala:65)
    at org.apache.spark.deploy.SparkHadoopUtil$$anon$2.run(SparkHadoopUtil.scala:64)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
    ... 4 more

当我在图形上设置spark. jars属性时，也在传递此jar位置

我们从应用程序创建的瓶子是胖子类型的，意味着它包含实际代码以及所有必需的依赖项，请参见下面的屏幕快照。

Answer 1

如果您查看日志，则会看到此内容

java" "-cp" "/opt/spark/spark-2.4.0/conf/:/opt/spark/spark-2.4.0/jars/*:/opt/hadoop/hadoop-3_1_1/etc/hadoop/"

除非每个Spark worker的/opt/spark/spark-2.4.0/jars/*文件夹中都包含gremlin JAR，否则您使用的类不存在。

推荐的用于特定应用程序的方法是Gradle Shadow插件，而不是--packages或spark.jars