Spark/Scala - Rename the Columns Produced by List

Question

How to rename the _1, _2 to meaningful column names in Spark/Scala?

root
   |-- aaa: string (nullable = true)
   |-- bbb: array (nullable = true)
   |    |-- element: struct (containsNull = true)
   |    |    |-- _1: string (nullable = true)
   |    |    |-- _2: long (nullable = false)

Answer 1

Create a case class and convert tuples into case class. this way you can have named tuple.

case class person(name:String,age:Int)
 val personRdd=rdd.map(_.split(<delimeter>)).map(x => person(x._1,x._2))

now you can access personRdd tuple values as

personRdd.map(p => (p.name+":"+p.age))