3

when we try to select Columns from DataFrame, one can use $"columnname" or col("columnname") or just "columnname".

My question is how $ symbol[which returns ColumnName] is working, i can understand i need to import sqlContext.implicits._ to use $ symbol on df.select

I dont see $ method on SQLImplicits class as well. I can see one method with the name symbolToColumn(scala.Symbol s).

Can someone explain more on this?

Shankar
  • 8,529
  • 26
  • 90
  • 159
  • Possible duplicate of [String interpolation in Scala 2.10 - How to interpolate a String variable?](http://stackoverflow.com/questions/13260864/string-interpolation-in-scala-2-10-how-to-interpolate-a-string-variable) – Sean Vieira Oct 24 '16 at 19:03

1 Answers1

5

It comes from StringToColumn implicit inner class in SQLImplicits (which is implemented by the implicits object).

StringContext is the way that f / s and other string interpolators are written in Scala.

Sean Vieira
  • 155,703
  • 32
  • 311
  • 293
  • Ha, so they are overloading the string interpolation behavior? How does Spark determine if you want string interpolation vs column selection? – evan.oman Oct 24 '16 at 19:04
  • 1
    If you wanted string interpolation you would use `s` / `f` and Scala would pull in the default string interpolator instead. – Sean Vieira Oct 24 '16 at 19:27
  • Ah, I see know. Thanks! – evan.oman Oct 24 '16 at 19:33
  • 6
    The *really* simple answer is that `foo"bla"` is syntactic sugar for `StringContext.foo("bla")`, and so you can "invent" your own string literal semantics just by providing a `StringContext.foo` method. – Jörg W Mittag Oct 24 '16 at 22:28