抽象方法错误在GraphLoader对象

Question

我已经创造了Graphx一个简单的项目，只要我试图运行这个测试项目中，我得到AbstractMethodError，误差这种方法edgeListFile里面正在添加，看起来像一些相关的记录，我是不是安倍看到的东西，请大家帮。

这里是我的.scala file

object graphtest extends App  {

  import org.apache.spark.graphx.{GraphLoader, VertexId}

    val spark = SparkSession.builder.master("local").appName("learning spark").getOrCreate
    val sc = spark.sparkContext


    val graph1 = GraphLoader.edgeListFile(spark.sparkContext, "E:\\code\\Cit-HepTh.txt")
    val res: (VertexId, Int) = graph1.inDegrees.reduce((a, b) => if (a._2 > b._2) a else b)

graph1.edges.collect().take(10).foreach(println)

}

这里是我的build.sbt文件

name := "myproject"

version := "0.1"

scalaVersion := "2.11.8"

mainClass in (Compile, packageBin) := Some("myproject.Processor")

libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-core" % "2.3.1",
  "org.apache.spark" %% "spark-sql" % "2.3.1",
  "org.scalatest" %% "scalatest" % "3.2.0-SNAP10" % Test,
  "com.typesafe" % "config" % "1.3.1",
  "org.apache.spark" %% "spark-mllib" % "2.0.1"
)

最后完整的栈跟踪失败

Exception in thread "main" java.lang.AbstractMethodError
at                 org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
at org.apache.spark.graphx.GraphLoader$.initializeLogIfNecessary(GraphLoader.scala:28)
at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
at org.apache.spark.graphx.GraphLoader$.log(GraphLoader.scala:28)
at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
at org.apache.spark.graphx.GraphLoader$.logInfo(GraphLoader.scala:28)
at org.apache.spark.graphx.GraphLoader$.edgeListFile(GraphLoader.scala:96)
at aaa.graphtest$.delayedEndpoint$zettasense$graphtest$1(Test.scala:15)
at aaa.graphtest$delayedInit$body.apply(Test.scala:6)
at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
at scala.App$class.main(App.scala:76)
at aaa.graphtest$.main(Test.scala:6)
at aaa.graphtest.main(Test.scala)

Answer 1

据库不匹配，我更新spark-core，spark-sql，spark-mlib到最新版本，它的工作顺利进行，这是我现在build.sbt看起来如何

name := "myproject"

version := "0.1"

scalaVersion := "2.11.8"

mainClass in(Compile, packageBin) := Some("myproject.Processor")


libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-core" % "2.4.0",
  "org.apache.spark" %% "spark-sql" % "2.4.0",
  "org.scalatest" %% "scalatest" % "3.2.0-SNAP10" % Test,
  "com.typesafe" % "config" % "1.3.1",
  "org.apache.spark" %% "spark-mllib" % "2.4.0"
)