我如何将参数传递给spark.sql（“”“”“）？

Question

我想将字符串传递给spark.sql

这是我的查询

mydf = spark.sql("SELECT * FROM MYTABLE WHERE TIMESTAMP BETWEEN '2020-04-01' AND '2020-04-08') 

我想传递一个字符串作为日期。

我尝试过此代码

val = '2020-04-08'

s"spark.sql("SELECT * FROM MYTABLE WHERE TIMESTAMP  BETWEEN $val  AND '2020-04-08'

Answer 1

尝试使用Python字符串格式{}和.format(val)，因为$val在scala中。

val = '2020-04-08'

spark.sql("SELECT * FROM MYTABLE WHERE TIMESTAMP  BETWEEN {}  AND '2020-04-08'".format(val)).show()

---

Example:

In Pyspark:

spark.sql("select * from tmp").show()
#+----+---+
#|name| id|
#+----+---+
#|   a|  1|
#|   b|  2|
#+----+---+

id='1'

spark.sql("select * from tmp where id={}".format(id)).show()
#+----+---+
#|name| id|
#+----+---+
#|   a|  1|
#+----+---+

---

In Scala:

使用string interpolation替换变量的值

val id=1
spark.sql(s"select * from tmp where id=$id").show()
//+----+---+
//|name| id|
//+----+---+
//|   a|  1|
//+----+---+