对kafka.KafkaUtils的依赖性

Question

我正在尝试将星光流与kafka集成在一起。我无法解决org.apache.spark.streaming.kafka.KafkaUtils的依赖性。以下是我的build.sbt：

name := "StreamingTest"

version := "1.0"

organization := "com.sundogsoftware"

scalaVersion := "2.12.10"

libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "3.0.0-preview2" % "provided",
"org.apache.spark" %% "spark-streaming-kafka-0-10" % "3.0.0-preview2",
"org.apache.spark" %% "spark-sql" % "3.0.0-preview2",
"org.apache.spark" %% "spark-streaming" % "3.0.0-preview2" % "provided",
"org.apache.kafka" %% "kafka" % "2.0.0"

)

我在项目中使用以下导入：

import org.apache.spark.{SparkConf, SparkContext, sql}
import org.apache.spark.sql.{SQLContext, SparkSession}
import org.apache.spark.streaming.{Seconds, StreamingContext}
import org.apache.spark.streaming.dstream.{DStream, InputDStream}
import org.apache.spark.streaming.kafka.KafkaUtils
import kafka.serializer.StringDecoder

[org.apache.spark.streaming.kafka.KafkaUtils以外的所有依赖关系都已解决。我正在使用Spark版本3.0.0-preview2和Scala版本2.12.10。

Answer 1

导入时正确的程序包名称是org.apache.spark.streaming.kafka010。