Griffin错误:java.lang.NoSuchMethodError

问题描述 投票:1回答:2

需要帮助。我正在尝试documentation中提供的这项工作。

我的系统中安装了以下版本:

  • Hadoop:3.2.0
  • Scala:2.13.1
  • 火花:3.0.0
  • Java:1.8.0
  • 配置单元:3.1.2

我正在提交spark作业以使用griffin来运行数据质量。我正在接受例外,您能否在这方面帮助我。

2020-04-04 08:37:35,456 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after waiting maxRegisteredResourcesWaitingTime: 30000000000(ns)
Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;
        at org.apache.griffin.measure.datasource.connector.batch.HiveBatchDataConnector.<init>(HiveBatchDataConnector.scala:47)
        at org.apache.griffin.measure.datasource.connector.DataConnectorFactory$$anonfun$getDataConnector$1.apply(DataConnectorFactory.scala:63)
        at org.apache.griffin.measure.datasource.connector.DataConnectorFactory$$anonfun$getDataConnector$1.apply(DataConnectorFactory.scala:62)
        at scala.util.Try$.apply(Try.scala:213)
        at org.apache.griffin.measure.datasource.connector.DataConnectorFactory$.getDataConnector(DataConnectorFactory.scala:61)
        at org.apache.griffin.measure.datasource.DataSourceFactory$$anonfun$1.apply(DataSourceFactory.scala:58)
        at org.apache.griffin.measure.datasource.DataSourceFactory$$anonfun$1.apply(DataSourceFactory.scala:57)
        at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:245)
        at scala.collection.immutable.List.foreach(List.scala:392)
        at scala.collection.TraversableLike.flatMap(TraversableLike.scala:245)
        at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:242)
        at scala.collection.immutable.List.flatMap(List.scala:355)
        at org.apache.griffin.measure.datasource.DataSourceFactory$.org$apache$griffin$measure$datasource$DataSourceFactory$$getDataSource(DataSourceFactory.scala:57)
        at org.apache.griffin.measure.datasource.DataSourceFactory$$anonfun$getDataSources$1.apply(DataSourceFactory.scala:40)
        at org.apache.griffin.measure.datasource.DataSourceFactory$$anonfun$getDataSources$1.apply(DataSourceFactory.scala:38)
        at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:245)
        at scala.collection.immutable.List.foreach(List.scala:392)
        at scala.collection.TraversableLike.flatMap(TraversableLike.scala:245)
        at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:242)
        at scala.collection.immutable.List.flatMap(List.scala:355)
        at org.apache.griffin.measure.datasource.DataSourceFactory$.getDataSources(DataSourceFactory.scala:38)
        at org.apache.griffin.measure.launch.batch.BatchDQApp$$anonfun$run$1.apply$mcZ$sp(BatchDQApp.scala:75)
        at org.apache.griffin.measure.launch.batch.BatchDQApp$$anonfun$run$1.apply(BatchDQApp.scala:67)
        at org.apache.griffin.measure.launch.batch.BatchDQApp$$anonfun$run$1.apply(BatchDQApp.scala:67)
        at scala.util.Try$.apply(Try.scala:213)
        at org.apache.griffin.measure.launch.batch.BatchDQApp.run(BatchDQApp.scala:67)
        at org.apache.griffin.measure.Application$.main(Application.scala:88)
        at org.apache.griffin.measure.Application.main(Application.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:928)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) 
java nosuchmethoderror
2个回答
0
投票
java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;
这绝对意味着您在应用程序中某个地方的Scala版本中存在冲突。

有一个非常相似的问题here。尝试遵循那里提供的答案(将Scala版本降级为2.11.x)。


0
投票
我认为没有为Scala 2.13编译/发行过任何Spark版本
© www.soinside.com 2019 - 2024. All rights reserved.