Scala Macros: Making a Map out of fields of a class in Scala

Question

Let's say that I have a lot of similar data classes. Here's an example class User which is defined as follows:

case class User (name: String, age: Int, posts: List[String]) {
  val numPosts: Int = posts.length

  ...

  def foo = "bar"

  ...
}

I am interested in automatically creating a method (at compile time) that returns a Map in a way that each field name is mapped to its value when it is called in runtime. For the example above, let's say that my method is called toMap:

val myUser = User("Foo", 25, List("Lorem", "Ipsum"))

myUser.toMap

should return

Map("name" -> "Foo", "age" -> 25, "posts" -> List("Lorem", "Ipsum"), "numPosts" -> 2)

How would you do this with macros?

Here's what I have done: First, I created a Model class as a superclass for all of my data classes and implemented the method in there like this:

abstract class Model {
  def toMap[T]: Map[String, Any] = macro toMap_impl[T]
}

class User(...) extends Model {
  ...
}

Then I defined a macro implementation in a separate Macros object:

object Macros {
  import scala.language.experimental.macros
  import scala.reflect.macros.Context
  def getMap_impl[T: c.WeakTypeTag](c: Context): c.Expr[Map[String, Any]] = {
    import c.universe._

    val tpe = weakTypeOf[T]

    // Filter members that start with "value", which are val fields
    val members = tpe.members.toList.filter(m => !m.isMethod && m.toString.startsWith("value"))

    // Create ("fieldName", field) tuples to construct a map from field names to fields themselves
    val tuples =
      for {
        m <- members
        val fieldString = Literal(Constant(m.toString.replace("value ", "")))
        val field = Ident(m)
      } yield (fieldString, field)

    val mappings = tuples.toMap

    /* Parse the string version of the map [i.e. Map("posts" -> (posts), "age" -> (age), "name" -> (name))] to get the AST
     * for the map, which is generated as:
     * 
     * Apply(Ident(newTermName("Map")), 
     *   List(
     *     Apply(Select(Literal(Constant("posts")), newTermName("$minus$greater")), List(Ident(newTermName("posts")))), 
     *     Apply(Select(Literal(Constant("age")), newTermName("$minus$greater")), List(Ident(newTermName("age")))), 
     *     Apply(Select(Literal(Constant("name")), newTermName("$minus$greater")), List(Ident(newTermName("name"))))
     *   )
     * )
     * 
     * which is equivalent to Map("posts".$minus$greater(posts), "age".$minus$greater(age), "name".$minus$greater(name)) 
     */
    c.Expr[Map[String, Any]](c.parse(mappings.toString))
  }
}

Yet I get this error from sbt when I try to compile it:

[error] /Users/emre/workspace/DynamoReflection/core/src/main/scala/dynamo/Main.scala:9: not found: value posts
[error]     foo.getMap[User]
[error]               ^

Macros.scala is being compiled first. Here is the snippet from my Build.scala:

lazy val root: Project = Project(
    "root",
    file("core"),
    settings = buildSettings
  ) aggregate(macros, core)

  lazy val macros: Project = Project(
    "macros",
    file("macros"),
    settings = buildSettings ++ Seq(
      libraryDependencies <+= (scalaVersion)("org.scala-lang" % "scala-reflect" % _))
  )

  lazy val core: Project = Project(
    "core",
    file("core"),
    settings = buildSettings
  ) dependsOn(macros)

What am I doing wrong? I think that the compiler tries to evaluate the field identifiers too when it creates the expression, but I don't know how to return them properly in the expression. Could you show me how to do that?

Thanks very much in advance.

Instead of using a macro, this may be easier http://stackoverflow.com/questions/1226555/case-class-to-map-in-scala — Noah, Jun 20 '13 at 20:47
@Noah, yep, seen that one already. But I'm interested in doing it in compile time with macros though. Thanks for the help! — Emre, Jun 20 '13 at 20:51
Instead of just `Ident(newTermName(posts))` you need to use `Select(c.prefix.tree, newTermName("posts"))`. — Eugene Burmako, Jun 20 '13 at 21:19
Thanks very much @EugeneBurmako! It works now. Could you elaborate a bit on why I needed to do that? Also please post it as an answer so I can select it. Thanks for your great work on macros too! — Emre, Jun 20 '13 at 21:28
Glad it helped! I think Travis Brown gave a much more comprehensive explanation, so I think it would be better if you accepted his answer. — Eugene Burmako, Jun 21 '13 at 05:59
You need to specify an explicit prefix for field selection, because macro expansion isn't evaluated in the context of an object (where one would have `this` available automatically), but is rather inlined into the call site. — Eugene Burmako, Jun 21 '13 at 06:02

Travis Brown · Accepted Answer · 2013-06-21T10:49:19.850

35

Note that this can be done much more elegantly without the toString / c.parse business:

import scala.language.experimental.macros

abstract class Model {
  def toMap[T]: Map[String, Any] = macro Macros.toMap_impl[T]
}

object Macros {
  import scala.reflect.macros.Context

  def toMap_impl[T: c.WeakTypeTag](c: Context) = {
    import c.universe._

    val mapApply = Select(reify(Map).tree, newTermName("apply"))

    val pairs = weakTypeOf[T].declarations.collect {
      case m: MethodSymbol if m.isCaseAccessor =>
        val name = c.literal(m.name.decoded)
        val value = c.Expr(Select(c.resetAllAttrs(c.prefix.tree), m.name))
        reify(name.splice -> value.splice).tree
    }

    c.Expr[Map[String, Any]](Apply(mapApply, pairs.toList))
  }
}

Note also that you need the c.resetAllAttrs bit if you want to be able to write the following:

User("a", 1, Nil).toMap[User]

Without it you'll get a confusing ClassCastException in this situation.

By the way, here's a trick that I've used to avoid the extra type parameter in e.g. user.toMap[User] when writing macros like this:

import scala.language.experimental.macros

trait Model

object Model {
  implicit class Mappable[M <: Model](val model: M) extends AnyVal {
    def asMap: Map[String, Any] = macro Macros.asMap_impl[M]
  }

  private object Macros {
    import scala.reflect.macros.Context

    def asMap_impl[T: c.WeakTypeTag](c: Context) = {
      import c.universe._

      val mapApply = Select(reify(Map).tree, newTermName("apply"))
      val model = Select(c.prefix.tree, newTermName("model"))

      val pairs = weakTypeOf[T].declarations.collect {
        case m: MethodSymbol if m.isCaseAccessor =>
          val name = c.literal(m.name.decoded)
          val value = c.Expr(Select(model, m.name))
          reify(name.splice -> value.splice).tree
      }

      c.Expr[Map[String, Any]](Apply(mapApply, pairs.toList))
    }
  }
}

Now we can write the following:

scala> println(User("a", 1, Nil).asMap)
Map(name -> a, age -> 1, posts -> List())

And don't need to specify that we're talking about a User.

edited Jun 21 '13 at 10:49

answered Jun 20 '13 at 21:45

Travis Brown

138,631
12
375
680

Why resetAllAttrs? Looks like it shouldn't be necessary here. – Eugene Burmako Jun 21 '13 at 05:58
It works without resetAllAttrs. Thanks for the great answer. One thing though, your implementation only outputs vals that are defined in the constructor (i.e. case accessor). I used isAccessor instead. I seem to have missed that method before. – Emre Jun 21 '13 at 06:56
Ah, right—I've removed the `resetAllAttrs` in the second example (it definitely is necessary in the first, though). And I wasn't sure about non-case class members, since `numPosts` doesn't appear in your desired output, for example. – Travis Brown Jun 21 '13 at 10:52
I need `resetAllAttrs` even in the second example -get a `ClassCastException` otherwise. – kiritsuku Jun 21 '13 at 11:24
@TravisBrown, forgot to add that. Fixed now. What would you recommend me to read to learn more about macros (apart from the ones on Scala website)? I'm still having a bit of hard time thinking what to do next while writing a macro. – Emre Jun 21 '13 at 11:36
This is very nice. Have you done the reverse also (Map to class)? – sourcedelica May 22 '14 at 19:55
@sourcedelica: All the key-value pairs would have to be literals to do the reverse at compile-time, which cuts down on the usefulness. – Travis Brown May 22 '14 at 20:00
@sourcedelica: You could of course generate (at compile-time) the code to parse the pairs at runtime and return an `Option[Foo]`, though. – Travis Brown May 22 '14 at 20:42
@TravisBrown: Since Case Class to Map is a trivial thing, why is this util not a part of std. lib ? – Sudheer Aedama Oct 09 '14 at 16:53
@Venkat: `Map[String, Any]` is just about the most antipattern-y of all the Scala antipatterns. Sometimes you have to do it, and in that case the macro is useful, but I'm glad it's not in the standard library or language. – Travis Brown Oct 09 '14 at 18:43
@TravisBrown: Exactly, *sometimes* you have to do it. I understand that it's harmful to have such anti-patterns in the std. lib, but hey what about many other things in the std. lib which are anti-patterns, yet serve a purpose for some limitations ? – Sudheer Aedama Oct 09 '14 at 20:02
@TravisBrown interaction between case class and string are obligatory donne with reflection? – crak Jun 13 '16 at 16:03

score 12 · Answer 2 · answered Dec 03 '14 at 09:00

12

There is an excellent blog post on map to/from case class conversion using macros.

answered Dec 03 '14 at 09:00

lisak

21,611
40
152
243

score 3 · Answer 3 · answered Oct 06 '18 at 15:59

Starting Scala 2.13, case classes (which are an implementation of Product) are now provided with a productElementNames method which returns an iterator over their field's names.

By zipping field names with field values obtained with productIterator one can obtained a Map out of whatever case class:

// val user = User("Foo", 25, List("Lorem", "Ipsum"))
(user.productElementNames zip user.productIterator).toMap
// Map[String, Any] = Map("name" -> "Foo", "age" -> 25, "posts" -> List("Lorem", "Ipsum"))

Scala Macros: Making a Map out of fields of a class in Scala

3 Answers3

Linked