Spark Dataframe concatenate strings

Question

How to concatenate Date and Time column (both string) using Spark-Scala

+---+-------------------+-----+
| ID|   Date            |Time
+---+------------------ +-----+
|  G|2014.01.14 00:00:00| 1:00
+---+-------------------+------

"2014.01.14 1:00"

you can do `df.withColumn("newCol", concat_ws(" ", col("Date"), col("Time")))` — Ramesh Maharjan, Jul 11 '18 at 10:48
Yes, I knew this approach, but what I don't know is that how to remove "00:00:00" from Date Column — milad ahmadi, Jul 11 '18 at 11:07

score 6 · Accepted Answer · answered Jul 11 '18 at 11:28

6

Try this,

result.withColumn("Date && Time", concat(regexp_replace(col("date"),"00:00:00",""), lit(""), (col("time"))))

answered Jul 11 '18 at 11:28

William R

739
2
13
34

score 1 · Answer 2 · answered Jul 11 '18 at 11:00

1

You can use :

result.withColumn("Date && Time", concat(col("Date"), lit(""), (col("Time")) ));

answered Jul 11 '18 at 11:00

YohanT

82
1
9

Spark Dataframe concatenate strings

2 Answers2