5

I have some questions about the interactions and relationships between DynamicMethods, Expression Trees and the DLR.

  1. I know that LambdaExpression.Compile uses an ILGenerator internally to create a Delegate. However, there are some fundamental differences between a compiled LambdaExpression and a DynamicMethod. For instance

    a. DynamicMethods invoke faster

    b. Compiled LambdaExpressions can embed closures (ConstantExpressions that are non primitive values)

    b. Compiled LambdaExpressions have no DeclaringType.

    Questions:

    a. Why are DynamicMethods faster to invoke than compiled LambdaExpressions?

    b. What's special about compiled LambdaExpressions that allows for closures? Does the Expression Tree actually generate a closure class when I use a non ConstantExpression? And if so, where does this generated class go?

    c. Where do compiled LambdaExpressions go (at runtime)? Where is the support for them implemented. It can't just be Reflection.Emit, can it?

  2. I know that the dynamic keyword is really just a compiler trick for emitting CSharp CallSites, Binders, etc. Internally, as I understand, these generate Expression Trees and also use a stripped down version of the C# compiler.

    Questions

    a. Are the expression trees being generated a function of CallSiteBinders in general or the specific implementation and usage of them in the Microsoft.CSharp dll?

    b. Are these expression trees composed of DynamicExpression nodes? Or something else? If something else, why?

    c. Where and why does the stripped down version of the C# compiler come into play? Why and how is it different than regular calls to LambdaExpression.Compile or DynamicMethods or any kind of IL generation? I can understand how CallSiteBinders could be used to build Expression Trees, but why is the C# compiler needed after that transformation occurs? And what does C# have to do with it at all once it's in the form of an Expression Tree (which is just an API).

Jeff
  • 35,755
  • 15
  • 108
  • 220
  • 1
    Why do you think `DynamicMethod` is faster? What do you base that claim on? – svick Jul 23 '13 at 08:05
  • 1
    Also, I think the two questions you're asking are not really related. You should probably ask them as two separate questions. – svick Jul 23 '13 at 08:08
  • I think they are faster by personal experience, and as cited here http://stackoverflow.com/questions/1296683/curiosity-why-does-expression-when-compiled-run-faster-than-a-minimal-dyna (even though the title indicates the opposite), here http://stackoverflow.com/questions/10673756/net-dynamic-method-best-performance. The difference is very marginal though, and I've always assumed it's just slight differences in the emitted IL. – Jeff Jul 23 '13 at 16:46
  • I asked them as one question because I want to understand if there is a relationship between the DLR and how expression trees are compiled and invoked. The original DLR (which was a separate dll) was really just an external implementation of expression trees (which included Expression.Dynamic support) that was compatible with .NET 2.0.... – Jeff Jul 23 '13 at 16:48
  • What I meant is that some people (like me) might know answers only to some of your questions. And if you have something like one great answer for question 1, one good answer for question 2 and one okay answer for both of them, the SO model pretty much breaks down. How do you then decide which one to accept or upvote? – svick Jul 23 '13 at 17:41

2 Answers2

4

I don't know much about dynamic, so I'm going to answer only the first part of your question.

Why are DynamicMethods faster to invoke than compiled LambdaExpressions?

I would be very surprised if they were, since Expression.Compile() internally uses DynamicMethod.

What's special about compiled LambdaExpressions that allows for closures? Does the Expression Tree actually generate a closure class when I use a non ConstantExpression? And if so, where does this generated class go?

That's easy to verify. Just look at Target and Method of a delegate generated from compiling an expression tree. You will notice that the Target (and first parameter of Method) is System.Runtime.CompilerServices.Closure. That's a class that contains a field object[] Constants, which is where non-primitive values from ConstantExpressions are stored.

Where do compiled LambdaExpressions go (at runtime)? Where is the support for them implemented. It can't just be Reflection.Emit, can it?

Like I said before, Expression.Compile() internally uses DynamicMethod. So, yeah, it is just Reflection.Emit.

svick
  • 236,525
  • 50
  • 385
  • 514
  • I see the bit about the closure class on the compiled lambda now. That answers a good deal of my question. Thanks. It also explains why JIT optimization could perform very differently for a regular dynamicmethod vs one created by an expression tree (it's bound to an instance of a Closure class) – Jeff Jul 23 '13 at 23:42
  • I disassembled the Closure class and more of the expression tree apis and I get it now (all of point number 1). I guess the easiest way to get to the bottom of my dlr question is to do some more disassembling. Thanks. I was missing the key bit about how the closure class kept track of constants. – Jeff Jul 24 '13 at 05:06
2

Well, I can't answer all your questions, but I can answer a few of them and I think that may answer most of yours. Perhaps at the minimum it will give you enough information to continue researching.

Why are DynamicMethods faster to invoke than compiled LambdaExpressions?

I don't think they are, perhaps you measured wrong and it's a JIT'ing difference

What's special about compiled LambdaExpressions that allows for closures? Does the Expression Tree actually generate a closure class when I use a non ConstantExpression? And if so, where does this generated class go?

This one I am unsure about. I would assume that Expression.Constant could contain reference types, then it's a non-issue, but if it can indeed only have value types, then I would guess that the compiler would just be generating an expression where the variables being captured in the closure are just passed in as parameters.

Where do compiled LambdaExpressions go (at runtime)? Where is the support for them implemented. It can't just be Reflection.Emit, can it?

System.Linq.Expressions is really just a friendlier API on top of Reflection.Emit, ergo they are just stored in memory just like Reflection.Emit is by default (though with Reflection.Emit you can save out the emitted code I believe)

Are the expression trees being generated a function of CallSiteBinders in general or the specific implementation and usage of them in the Microsoft.CSharp dll?

I've only done a little System.Dynamic work, so I cannot answer this, but it's been my understanding that the CallSiteBinder just caches and invokes the expression, but passes the actual generation off to something else (ie DynamicObject). But again, you likely know more here than I do.

Are these expression trees composed of DynamicExpression nodes? Or something else? If something else, why?

No, dynamic is still bound by the same rules as everything else in .net. Dynamic is just saying "at runtime, when i do x, go and try to build the code that I would have normally written and execute it for me." Things like DynamicObject are just going to build a plain old expression tree, dynamic object just provides you some meta data so that you can in fact build that tree (like return type, type of access, name, etc).

Where and why does the stripped down version of the C# compiler come into play? Why and how is it different than regular calls to LambdaExpression.Compile or DynamicMethods or any kind of IL generation? I can understand how CallSiteBinders could be used to build Expression Trees, but why is the C# compiler needed after that transformation occurs? And what does C# have to do with it at all once it's in the form of an Expression Tree (which is just an API).

I'm not sure what you mean by stripped down compiler or what the runtime actually does to generate the IL code (which I think is what you are after here), so I don't think I can answer this.

I will say, however, like I said before: the System.Linq.Expression stuff is really just a friendly API on top of Reflection.Emit. The expression tree is just the information it needs to go to reflection.emit, generate a dynamic method, and return it back to you.

Darren Kopp
  • 76,581
  • 9
  • 79
  • 93
  • The stripped down version of the compiler reference is based off a comment made by Eric Lippert. Should have noted that in my question, sorry. – Jeff Jul 23 '13 at 23:39