Closed-over variables in LambdaBuilder

Dec 11, 2008 at 3:45 AM
Imagine compiling a Lisp variant. You're handed an AST in the form of an undecorated Lisp structure. You'd like to do only one pass over the AST.

Consider the form: <tt>(fn [a] (fn [b] (+ a b) )</tt> --<tt>fn</tt>== lambda -- returning a lambda expression with the outer parameter <tt>a</tt> closed over.

I can hand-craft this easily (tossing in some gratuitous <tt>int</tt> typing to simplify matters):
<tt>LambdaBuilder outerLb = Utils.Lambda(typeof(Delegate), "ted");
outerLb.DoNotAddContext = true;

ParameterExpression a = outerLb.ClosedOverParameter(typeof(int), "a");

LambdaBuilder innerLb = Utils.Lambda(typeof(int), "fred");
innerLb.DoNotAddContext = true;

ParameterExpression b = Expression.Parameter(typeof(int),"b");

Expression exp = Expression.Add(a, b);
innerLb.Body = exp;
innerLb.Parameters.Add(b);
LambdaExpression innerLambda = innerLb.MakeLambda();

outerLb.Body = innerLambda;
LambdaExpression outerLambda = outerLb.MakeLambda();

</tt>
The problem is that I can't figure out how to do this in one pass. I do not see any way to make the <tt>a</tt> parameter closed except by creating it using <tt>LambdaBuilder.ClosedOverParameter</tt>. That means I have to know at the time of parameter creation--which needs to be before descending into the nested form in any reasonable approach--that this parameter is to be closed over, and the only way to know that is to first visit the subform. Hence, two passes.

Given that the only real difference between <tt>ClosedOverParameter</tt> and <tt>Parameter</tt> is the equivalent of the setting of a flag, it would be nice if one could do just that: create the outer param, parse the subform and note the non-local variable reference, and set the closed-over flag on the way back out.

Coordinator
Dec 11, 2008 at 6:56 PM

I think you’ll find this doesn’t matter for you.  “Closed over” here I believe is really meaning visible to the CodeContext.  Because you’re setting DoNotAddContext = true you’ll never encounter the code path in LambdaBuilder that actually does anything with the flag that gets set so calling either one should have the same effect.

Generally speaking you can just create parameters and refer to them from an inner lambda and the DLR will just make it work.  You could even create the lambda expressions directly with Expression.Lambda and avoid LambdaBuilder entirely.  That might make it more clear what the DLR is actually doing under the hood.

From: dmiller [mailto:notifications@codeplex.com]
Sent: Wednesday, December 10, 2008 7:45 PM
To: Dynamic Language Runtime
Subject: Closed-over variables in LambdaBuilder [dlr:42019]

From: dmiller

Imagine compiling a Lisp variant. You're handed an AST in the form of an undecorated Lisp structure. You'd like to do only one pass over the AST.

Consider the form: <tt>(fn [a] (fn [b] (+ a b) )</tt> --<tt>fn</tt>== lambda -- returning a lambda expression with the outer parameter <tt>a</tt> closed over.

I can hand-craft this easily (tossing in some gratuitous <tt>int</tt> typing to simplify matters):

<tt>LambdaBuilder outerLb = Utils.Lambda(typeof(Delegate), "ted");
outerLb.DoNotAddContext = true;

ParameterExpression a = outerLb.ClosedOverParameter(typeof(int), "a");

LambdaBuilder innerLb = Utils.Lambda(typeof(int), "fred");
innerLb.DoNotAddContext = true;

ParameterExpression b = Expression.Parameter(typeof(int),"b");

Expression exp = Expression.Add(a, b);
innerLb.Body = exp;
innerLb.Parameters.Add(b);
LambdaExpression innerLambda = innerLb.MakeLambda();

outerLb.Body = innerLambda;
LambdaExpression outerLambda = outerLb.MakeLambda();

 </tt>

The problem is that I can't figure out how to do this in one pass. I do not see any way to make the <tt>a</tt> parameter closed except by creating it using <tt>LambdaBuilder.ClosedOverParameter</tt>. That means I have to know at the time of parameter creation--which needs to be before descending into the nested form in any reasonable approach--that this parameter is to be closed over, and the only way to know that is to first visit the subform. Hence, two passes.

Given that the only real difference between <tt>ClosedOverParameter</tt> and <tt>Parameter</tt> is the equivalent of the setting of a flag, it would be nice if one could do just that: create the outer param, parse the subform and note the non-local variable reference, and set the closed-over flag on the way back out.

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Dec 11, 2008 at 8:33 PM

Like Dino said, it’s best to use Expression.Lambda directly if you can—LambdaBuilder will probably not stay around in its current form. LambdaExpression.Compile binds closures automatically, but they are bound by ParameterExpression instance, not name. So when you’re generating the expression tree, you’ll need to track a Dictionary<string, ParameterExpression> that maps the variable’s name to its corresponding node.

- John

From: dinov [mailto:notifications@codeplex.com]
Sent: Thursday, December 11, 2008 10:57 AM
To: John Messerly
Subject: Re: Closed-over variables in LambdaBuilder [dlr:42019]

From: dinov

I think you’ll find this doesn’t matter for you. “Closed over” here I believe is really meaning visible to the CodeContext. Because you’re setting DoNotAddContext = true you’ll never encounter the code path in LambdaBuilder that actually does anything with the flag that gets set so calling either one should have the same effect.

Generally speaking you can just create parameters and refer to them from an inner lambda and the DLR will just make it work. You could even create the lambda expressions directly with Expression.Lambda and avoid LambdaBuilder entirely. That might make it more clear what the DLR is actually doing under the hood.

From: dmiller [mailto:notifications@codeplex.com]
Sent: Wednesday, December 10, 2008 7:45 PM
To: Dynamic Language Runtime
Subject: Closed-over variables in LambdaBuilder [dlr:42019]

From: dmiller

Imagine compiling a Lisp variant. You're handed an AST in the form of an undecorated Lisp structure. You'd like to do only one pass over the AST.

Consider the form: <tt>(fn [a] (fn [b] (+ a b) )</tt> --<tt>fn</tt>== lambda -- returning a lambda expression with the outer parameter <tt>a</tt> closed over.

I can hand-craft this easily (tossing in some gratuitous <tt>int</tt> typing to simplify matters):

<tt>LambdaBuilder outerLb = Utils.Lambda(typeof(Delegate), "ted");

outerLb.DoNotAddContext = true;



ParameterExpression a = outerLb.ClosedOverParameter(typeof(int), "a");



LambdaBuilder innerLb = Utils.Lambda(typeof(int), "fred");

innerLb.DoNotAddContext = true;



ParameterExpression b = Expression.Parameter(typeof(int),"b");



Expression exp = Expression.Add(a, b);

innerLb.Body = exp;

innerLb.Parameters.Add(b);

LambdaExpression innerLambda = innerLb.MakeLambda();



outerLb.Body = innerLambda;

LambdaExpression outerLambda = outerLb.MakeLambda();



 </tt>

The problem is that I can't figure out how to do this in one pass. I do not see any way to make the <tt>a</tt> parameter closed except by creating it using <tt>LambdaBuilder.ClosedOverParameter</tt>. That means I have to know at the time of parameter creation--which needs to be before descending into the nested form in any reasonable approach--that this parameter is to be closed over, and the only way to know that is to first visit the subform. Hence, two passes.

Given that the only real difference between <tt>ClosedOverParameter</tt> and <tt>Parameter</tt> is the equivalent of the setting of a flag, it would be nice if one could do just that: create the outer param, parse the subform and note the non-local variable reference, and set the closed-over flag on the way back out.

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Dec 12, 2008 at 5:12 AM
Well, that certainly simplifies things.  I missed the DoNotAddContext switch on the generation.  LambdaBuilder about to be excised.  Thanks much. -- dm