scala数据帧中的rlike出现错误

问题描述 投票:0回答:1

我正在尝试将下面的Hive SQL语句转换为Spark数据帧,但出现错误。

case when (lower(message_txt) rlike '.*sampletext(\\s?is\\s?)newtext.*' ) then 'P' else 'Y'

样本数据:message_txt = "This is new sampletext, followed by newtext"

[请帮助我提供等效的spark数据框语句。

apache-spark apache-spark-sql rlike
1个回答
0
投票

在sql中end的末尾添加[case statement

Example:

In spark Sql:

val df=Seq(("This is new sampletext, followed by newtext")).toDF("message_txt")
df.createOrReplaceTempView("tmp")
spark.sql("select case when (lower(message_txt) rlike '.sampletext(\\s?is\\s?)newtext.' ) then 'P' else 'Y' end from tmp").show()
//+--------------------------------------------------------------------------------+
//|CASE WHEN lower(message_txt) RLIKE .sampletext(s?iss?)newtext. THEN P ELSE Y END|
//+--------------------------------------------------------------------------------+
//|                                                                               Y|
//+--------------------------------------------------------------------------------+

In dataframe API:

df.withColumn("status", when(lower(col("message_txt")).rlike(".sampletext(\\s?is\\s?)newtext."),"P").otherwise("Y")).show()
//+--------------------+------+
//|         message_txt|status|
//+--------------------+------+
//|This is new sampl...|     Y|
//+--------------------+------+
© www.soinside.com 2019 - 2024. All rights reserved.