Where can I learn about constructing AST's for Scala macros?

Question

Where I can learn how to construct the AST's that Scala's macros generate?

The Scaladoc isn't as helpful as I'd like. For example:

abstract def Apply(sym: Universe.Symbol, args: Universe.Tree*): Universe.Tree
A factory method for Apply nodes.

But how do I figure out what an Apply node is? Where can I find a list of the node types in AST's, and how they fit together?

score 40 · Accepted Answer · edited May 23 '17 at 12:25

There isn't a lot of documentation for the internals of the compiler available, but the things that are available should be enough to get started.

Mirko Stocker, has written his Master Thesis about Scala Refactoring. In Appendix D (p. 95) he describes the architecture of the AST. It includes also a graphical overview:

Scala AST

Another way to find information about the AST is to look directly into the sources of reflect.internal.Trees, which contains the AST.

If one needs to find out how a specific source code snippet is represented internally there is reify:

scala> import reflect.runtime.universe._
import reflect.runtime.universe._

scala> showRaw(reify{val i = 0}.tree)
res8: String = Block(List(ValDef(Modifiers(), newTermName("i"), TypeTree(),
  Literal(Constant(0)))), Literal(Constant(())))

Thanks! That thesis is a great resource. – Bill Feb 09 '13 at 21:57 — Bill, Feb 09 '13 at 21:57
How can we draw such a graph? – Freewind Feb 28 '15 at 15:02 — Freewind, Feb 28 '15 at 15:02

score 22 · Answer 2 · answered Feb 10 '13 at 08:30

You could take a look at the scaladoc (http://docs.scala-lang.org/overviews/reflection/symbols-trees-types.html#trees) or at the slides (http://scalamacros.org/talks/2012-04-28-MetaprogrammingInScala210.pdf, the "Learn to learn" part).

Here's what I usually do. I wrote a simple script called parse, which takes Scala code as an argument and then compiles it with -Xprint:parser -Ystop-after:parser -Yshow-trees-stringified -Yshow-trees-compact (parse uses another helper script: adhoc-scalac. click here to see its sources as well).

The advantage this approach has over showRaw is that it doesn't require the code to typecheck. You could write a small snippet of code, which refers to non-existent variables or classes, and it still will successfully run and show you the AST. Here's an example of output:

09:26 ~$ parse 'class C { def x = 2 }'
[[syntax trees at end of parser]]// Scala source: tmp36sVGp
package <empty> {
  class C extends scala.AnyRef {
    def <init>() = {
      super.<init>();
      ()
    };
    def x = 2
  }
}
PackageDef(Ident(TermName("<empty>")), List(ClassDef(Modifiers(), TypeName("C"), List(), Template(List(Select(Ident(scala), TypeName("AnyRef"))), emptyValDef, List(DefDef(Modifiers(), nme.CONSTRUCTOR, List(), List(List()), TypeTree(), Block(List(pendingSuperCall), Literal(Constant(())))), DefDef(Modifiers(), TermName("x"), List(), List(), TypeTree(), Literal(Constant(2))))))))

There's also a script called typecheck, which does the same, but stops after typer. That's sometimes useful to understand how exactly the typechecker transforms the parser trees. However, both toolboxes and macros work with parser trees, so I use typecheck for tree construction purposes very rarely.

Thanks, Eugene! The scalac call is very helpful. – Bill Feb 12 '13 at 03:15 — Bill, Feb 12 '13 at 03:15

Where can I learn about constructing AST's for Scala macros?

2 Answers2

Linked