how to pivot Spark dataframe table?

Question

I have this table of 3 columns:

+---+----+----+
| id|type| val|
+---+----+----+
|  1|   A|   0|
|  2|   A|   0|
|  4|   A|   0|
|  2|   B|   1|
|  4|   B|   1|
+---+----+----+

and I would like to transforme it to something like:

+---+----+----+
|   |   A|   B|
+---+----+----+
|  1|   0|   -|
|  2|   1|   1|
|  4|   0|   1|
+---+----+----+

I tried this but didn't work:

val data_array = data.pivot(cols=['type'],rows=['id'],values='val')

Answer 1

df.groupBy("id").pivot("type").agg(first("value")).na.fill("-").show

df is the dataframe created from the test data file