如何测试返回Dataset [String,T]的函数

问题描述 投票:-1回答:1

你好,我有以下函数返回一个数据集[(String,SubsidiariesImpactedStage)]:

def reduceSubsidiaries(dsSubsidiaries: Dataset[SubsidiariesImpactedStage]): Dataset[(String, SubsidiariesImpactedStage)] = {}

SubsidiariesImpactedStage以下

case class SubsidiariesImpactedStage(
  event_timestamp: Timestamp,
  event_type: String,
  subsidiary_uuid: String,
  subsidiary_id: Long,
  company_uuid: String
)

这是我写的以下测试

  val ts1 = Timestamp.valueOf("2019-08-01 00:00:00")
      val ts2 = Timestamp.valueOf("2019-09-20 00:00:00")
      val ts3 = Timestamp.valueOf("2019-11-27 00:00:00")
      val subsidiaries = Seq(
        SubsidiariesImpactedStage(ts1,"active","sub_uuid1",32L,"comp_uuid1"),
        SubsidiariesImpactedStage(ts2, "inactive","sub_uuid1",32L, "comp_uuid1"),
        SubsidiariesImpactedStage(ts3, "active", "sub_uuid1", 5L, "latest_comp_uuid1")
      ).toDS()

      val actual:Dataset[(String, SubsidiariesImpactedStage)] = reduceSubsidiaries(subsidiaries)
      val expected: Dataset[(String, SubsidiariesImpactedStage)] = Seq(
        SubsidiariesImpactedStage(ts3,"active","sub_uuid1", 5L, "latest_comp_uuid1")
      ).toDS()

我在expected上遇到错误,因为它是Dataset[SubsidiaresImpactedStage]而不是Dataset[(String, SubsidiariesImpactedStage)],我该如何解决?

scala apache-spark apache-spark-sql dataset apache-spark-dataset
1个回答
0
投票

最后一个命令的左侧从元组expected开始定义Dataset[(String, SubsidiariesImpactedStage)]

右侧分配Dataset[SubsidiariesImpactedStage]

尝试将右侧更改为

val expected: Dataset[(String, SubsidiariesImpactedStage)] = Seq(("myString",
  SubsidiariesImpactedStage("active","sub_uuid1", 5L, "latest_comp_uuid1")
  )
).toDS()

"myString"是您认为与expected相关的任何字符串。

© www.soinside.com 2019 - 2024. All rights reserved.