DLR 1.0 ExpressionType.Assign replaced?

Jul 29, 2010 at 3:54 PM
I'm trying to create the code behind an expression like:
    x = 5;
and the docs say I need to use BinaryOperation with ExpressionType.Assign. This is my code:
        public Expression AssignExpression(Expression v1, Expression v2)
        {
            Expression result;
            DynamicMetaObjectBinder binder = languageContext.CreateBinaryOperationBinder(ExpressionType.Assign);
            result = DynamicExpression.Dynamic(binder, typeof(object), v1, v2);
            return result;
        }
This is failing at compile time (not run time) with the message:
[System.ArgumentException]
	{"Invalid argument value\r\nParameter name: operation"}
If I replace ExpressionType.Assign with ExpressionType.Add then the compile succeeds and the Expression tree is valid.

I interpret this to mean that BinaryOperation does not accept ExpressionType.Assign. What should I be using here to create an assignment operation?

Jul 29, 2010 at 4:05 PM

Assignment is not a potentially dynamic operation, so there's no place for the Dynamic node here. Just use Expression.Assign directly.

Jul 29, 2010 at 4:20 PM
Thanks for responding. I have tried that, but I get an error at compile time:
    {"Expression must be writeable\r\nParameter name: left"}	System.Exception {System.ArgumentException}
The LHS is a Dynamic global variable reference, so it shows up in the debugger as:
    {GetMember x(0)}
As I understand the docs, it is legal to put a GetMember on the LHS of Assign. Clearly I'm missing something.

My assign builder looks like this now:

        public Expression AssignExpression(Expression v1, Expression v2)
        {
            Expression result;
            result = BinaryExpression.Assign(v1, v2);
            return result;
        }
and the v1 input is the output of this:
        public Expression VariableExpression(string ident)
        {
            // use a fake target as a placeholder for the global scope
            Expression subject = Expression.Constant(0, typeof(int));
            DynamicMetaObjectBinder binder = languageContext.CreateGetMemberBinder(ident, false);
            result = DynamicExpression.Dynamic(binder, typeof(object), subject);
            return result;
        }
Jul 29, 2010 at 4:30 PM

It's legal to have a GetMember as an lvalue if the GetMember is a reference to a field or property because those can be resolved statically. It doesn't make sense to have a dynamic GetMember as an lvalue -- for that to work, you'd effectively need GetMember to return a reference. Or there would have to have been a dynamic assignment node (and I'm not going to try to convince you that that would be a bad idea because -- whether or not it is, it's not currently present).

I don't quite understand how you're trying to construct your variable scope, but it's best to keep it simple. Assuming you're compiling each mathematical expression as it's being typed by the user, you'll need some kind of dictionary to map variable names onto values. A ScriptScope is good for this. VariableExpressions aren't the right way to carry values from one compilation to the next. A source level "x = 5" should then translate into code that generates something like scope.SetValue("x", 5) and a get from the scope would need a similar transformation.

Jul 29, 2010 at 4:45 PM
Again, thanks for replying. I'm not sure how to get where I want to be. My goal is to create an interpreter where a script expression can be compiled early, then executed one or more times at a later date. In order for that to work, an assignment like "x = 5" cannot resolve "x" during compile time at all. It simply has to create an Expression that will tell the executor to perform a lookup and an assignment to "x" later. I have no idea how to do that.

I think I understand the concept of building the Expression tree, but every time I use the direct Expression instead of Expression.Dynamic, the compiler wants to resolve types and variable bindings at compile time instead of simply constructing a tree that would make the resolution happen later at execution time.

Have I simply missed a setting where I can tell the compiler that the target language is late bound and dynamically typed? If I can tell the compiler to defer variable resolution to run time, I'm happy.

In general, resolving "x = 5" to 'scope.SetValue("x",5)' will work for simple variables. From the docs it looks like I should be able to use an Assign mechanism that internally can handle "x[0] = 5" and "x.y = 5" through the same code path (that is, Variable, Index and Member assignment handled by the BinaryOperation). Ultimately I will want to support these as well. I thought that was what ExpressionType.Assign was supposed to do for me.

Jul 29, 2010 at 4:55 PM
Oh, and as a further refinement, I would like to be able to support a member reference like this:
    x.y.("parm" + n).z = 5;
Which would compute the expression in parentheses as a name of a member. Maybe the syntax is dumb, but the computed member reference could be useful. I think it still boils down to resolving variable names and values at run time instead of compile time.
Coordinator
Jul 29, 2010 at 5:12 PM

Interestingly, if you looked at the sympl test file (python version) or Main (C# version), you'd see it drops into a little interpreter. If you looked at the sympl code and threw out everything related to control flow, creating lambdas, function defs, etc., you'd pretty much be left with its global variables and simple expressions. You could also probably pick up on its dotted expression compilation to implement your computed name lookup (you'd use a dynamic get member just like sympl, but you'd add an expression to compute the name.

I do get you have a bunch of working code, and matching the two may not be trivial; it might give you some lift though.

Bill

From: asthomas [mailto:notifications@codeplex.com]
Sent: Thursday, July 29, 2010 9:56 AM
To: Bill Chiles
Subject: Re: DLR 1.0 ExpressionType.Assign replaced? [dlr:221868]

From: asthomas

Oh, and as a further refinement, I would like to be able to support a member reference like this:

 
    x.y.("parm" + n).z = 5;

Which would compute the expression in parentheses as a name of a member. Maybe the syntax is dumb, but the computed member reference could be useful. I think it still boils down to resolving variable names and values at run time instead of compile time.

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Jul 29, 2010 at 5:51 PM
Thanks, Bill. I'll dig a little. I'm not bothered by rewriting code. At this stage I'm trying to assess how to use the DLR to create a substantial scripting language. So my test case is a simple calculator with global registers, but my long-term goal is a dynamically scoped, dynamically typed interpreter that has access to the C# built-in objects and methods.

As I'm writing this, I realize that I've been reading the code from the wrong Sympl implementation. I know that makes me look like an idiot - *sigh*. I'll go back and try to figure out what the codeplex version is doing.

Jul 29, 2010 at 6:27 PM
Nope, that didn't help me. Sympl appears to be statically resolving its identifiers. I still have no clue as to how I would construct my assignment expression so that it resolves at execution time instead of at compile time. I understand (somewhat) in principle, but I need a little help on how it could actually be done.

It boils down to this: How can I make or wrap an Expression.Assign() whose LHS is run-time computed? If Expression.Assign cannot do it, what could? I suspect I could make a Lambda expression that will call my own function at runtime, but I'm not sure if that's overkill or exactly how to specify such a thing.

Coordinator
Jul 29, 2010 at 6:54 PM

You won’t use Expression.Assign for this. What you need to do is have some place to store the variables (e.g. a dictionary) and then you need to make all of the read/writes to the variables become read/writes to the dictionary. For example let’s consider the global code such as:

a = 42

print a

For which you want to generate a .NET method which looks something like this C# code:

public void OuterCode(Dictionary<string, object> variables) {

variables[“a”] = 42;

print variables[“a”];

}

Which would be constructed like:

var dict = Expression.Parameter(typeof(Dictionary<string, object>), "variables");

var lambda = Expression.Lambda(

Expression.Block(

Expression.Call(

dict,

typeof(Dictionary<string, object>).GetMethod("set_Item"),

Expression.Constant("a"),

Expression.Convert(Expression.Constant(42), typeof(object))

),

Expression.Call(

typeof(LanguageOps).GetMethod("Print"),

Expression.Call(

dict,

typeof(Dictionary<string, object>).GetMethod("get_Item"),

Expression.Constant("a")

)

)

),

"OuterCode",

new[] { dict }

);

If you really want something like an Assignment node you can start using reducible nodes to fancy this up and make it a little easier to generate the code. For example first you can create a reducible node for referring to globals:

class MyGlobal : Expression {

private readonly string _name;

private readonly Expression _dict;

public MyGlobal(Expression dict, string name) {

_dict = dict;

_name = name;

}

public string Name {

get { return _name; }

}

public Expression Dict {

get { return _dict; }

}

public override Expression Reduce() {

return Expression.Call(

_dict,

typeof(Dictionary<string, object>).GetMethod("get_Item"),

Expression.Constant(_name)

);

}

public override ExpressionType NodeType {

get { return ExpressionType.Extension; }

}

public override bool CanReduce {

get { return true; }

}

public override Type Type {

get { return typeof(object); }

}

}

Then you can create another reducible node for assignment that supports the normal ET assignments plus an assignment to your global:

class MyAssign : Expression {

private readonly Expression _lhs, _rhs;

public MyAssign(Expression lhs, Expression rhs) {

_lhs = lhs;

_rhs = rhs;

}

public override Expression Reduce() {

if (_lhs is ParameterExpression || _lhs is MemberExpression || _lhs is IndexExpression) {

return Expression.Assign(_lhs, _rhs);

} else if (_lhs is MyGlobal) {

MyGlobal glob = _lhs as MyGlobal;

return Expression.Call(

glob.Dict,

typeof(Dictionary<string, object>).GetMethod("set_Item"),

Expression.Constant(glob.Name),

Expression.Convert(_rhs, typeof(object))

);

}

throw new InvalidOperationException();

}

public override ExpressionType NodeType {

get { return ExpressionType.Extension; }

}

public override bool CanReduce {

get { return true; }

}

public override Type Type {

get { return _rhs.Type; }

}

}

And now to create the lambda it looks like:

var dict = Expression.Parameter(typeof(Dictionary<string, object>), "variables");

var lambda = Expression.Lambda(

Expression.Block(

new MyAssign(

new MyGlobal(dict, "a"),

Expression.Constant(42)

),

Expression.Call(

typeof(LanguageOps).GetMethod("Print"),

new MyGlobal(dict, "a")

)

),

"OuterCode",

new[] { dict }

);

Now if you want to support your lookup of a name by a string you can have something like:

class MyDynamicGlobal : Expression {

private readonly Expression _name;

private readonly Expression _dict;

public MyDynamicGlobal(Expression dict, Expression name) {

_dict = dict;

_name = name;

}

public Expression Name {

get { return _name; }

}

public Expression Dict {

get { return _dict; }

}

public override Expression Reduce() {

return Expression.Call(

_dict,

typeof(Dictionary<string, object>).GetMethod("get_Item"),

Expression.Convert(_name, typeof(string));

);

}

public override ExpressionType NodeType {

get { return ExpressionType.Extension; }

}

public override bool CanReduce {

get { return true; }

}

public override Type Type {

get { return typeof(object); }

}

}

Note all this code is untested but should give you the general idea of what to do.

From: asthomas [mailto:notifications@codeplex.com]
Sent: Thursday, July 29, 2010 11:28 AM
To: Dino Viehland
Subject: Re: DLR 1.0 ExpressionType.Assign replaced? [dlr:221868]

From: asthomas

Nope, that didn't help me. Sympl appears to be statically resolving its identifiers. I still have no clue as to how I would construct my assignment expression so that it resolves at execution time instead of at compile time. I understand (somewhat) in principle, but I need a little help on how it could actually be done.

It boils down to this: How can I make or wrap an Expression.Assign() whose LHS is run-time computed? If Expression.Assign cannot do it, what could? I suspect I could make a Lambda expression that will call my own function at runtime, but I'm not sure if that's overkill or exactly how to specify such a thing.

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Coordinator
Jul 29, 2010 at 7:29 PM

I realize it is hard to tease out of the Sympl code, but it does dynamically bind to globals. In AnalyzeAssignExpr, you'll see in the SymplIdExpr branch where it calls scope.GetModuleExpr to create a dynamic expression SetMember using the ID and the module globals. This is similar to Dino's OuterCode's dictionary. The document explains the globals stuff.

Bill

From: dinov [mailto:notifications@codeplex.com]
Sent: Thursday, July 29, 2010 11:55 AM
To: Bill Chiles
Subject: Re: DLR 1.0 ExpressionType.Assign replaced? [dlr:221868]

From: dinov

You won’t use Expression.Assign for this. What you need to do is have some place to store the variables (e.g. a dictionary) and then you need to make all of the read/writes to the variables become read/writes to the dictionary. For example let’s consider the global code such as:

a = 42

print a

For which you want to generate a .NET method which looks something like this C# code:

public void OuterCode(Dictionary<string, object> variables) {

variables[“a”] = 42;

print variables[“a”];

}

Which would be constructed like:

var dict = Expression.Parameter(typeof(Dictionary<string, object>), "variables");

var lambda = Expression.Lambda(

Expression.Block(

Expression.Call(

dict,

typeof(Dictionary<string, object>).GetMethod("set_Item"),

Expression.Constant("a"),

Expression.Convert(Expression.Constant(42), typeof(object))

),

Expression.Call(

typeof(LanguageOps).GetMethod("Print"),

Expression.Call(

dict,

typeof(Dictionary<string, object>).GetMethod("get_Item"),

Expression.Constant("a")

)

)

),

"OuterCode",

new[] { dict }

);

If you really want something like an Assignment node you can start using reducible nodes to fancy this up and make it a little easier to generate the code. For example first you can create a reducible node for referring to globals:

class MyGlobal : Expression {

private readonly string _name;

private readonly Expression _dict;

public MyGlobal(Expression dict, string name) {

_dict = dict;

_name = name;

}

public string Name {

get { return _name; }

}

public Expression Dict {

get { return _dict; }

}

public override Expression Reduce() {

return Expression.Call(

_dict,

typeof(Dictionary<string, object>).GetMethod("get_Item"),

Expression.Constant(_name)

);

}

public override ExpressionType NodeType {

get { return ExpressionType.Extension; }

}

public override bool CanReduce {

get { return true; }

}

public override Type Type {

get { return typeof(object); }

}

}

Then you can create another reducible node for assignment that supports the normal ET assignments plus an assignment to your global:

class MyAssign : Expression {

private readonly Expression _lhs, _rhs;

public MyAssign(Expression lhs, Expression rhs) {

_lhs = lhs;

_rhs = rhs;

}

public override Expression Reduce() {

if (_lhs is ParameterExpression || _lhs is MemberExpression || _lhs is IndexExpression) {

return Expression.Assign(_lhs, _rhs);

} else if (_lhs is MyGlobal) {

MyGlobal glob = _lhs as MyGlobal;

return Expression.Call(

glob.Dict,

typeof(Dictionary<string, object>).GetMethod("set_Item"),

Expression.Constant(glob.Name),

Expression.Convert(_rhs, typeof(object))

);

}

throw new InvalidOperationException();

}

public override ExpressionType NodeType {

get { return ExpressionType.Extension; }

}

public override bool CanReduce {

get { return true; }

}

public override Type Type {

get { return _rhs.Type; }

}

}

And now to create the lambda it looks like:

var dict = Expression.Parameter(typeof(Dictionary<string, object>), "variables");

var lambda = Expression.Lambda(

Expression.Block(

new MyAssign(

new MyGlobal(dict, "a"),

Expression.Constant(42)

),

Expression.Call(

typeof(LanguageOps).GetMethod("Print"),

new MyGlobal(dict, "a")

)

),

"OuterCode",

new[] { dict }

);

Now if you want to support your lookup of a name by a string you can have something like:

class MyDynamicGlobal : Expression {

private readonly Expression _name;

private readonly Expression _dict;

public MyDynamicGlobal(Expression dict, Expression name) {

_dict = dict;

_name = name;

}

public Expression Name {

get { return _name; }

}

public Expression Dict {

get { return _dict; }

}

public override Expression Reduce() {

return Expression.Call(

_dict,

typeof(Dictionary<string, object>).GetMethod("get_Item"),

Expression.Convert(_name, typeof(string));

);

}

public override ExpressionType NodeType {

get { return ExpressionType.Extension; }

}

public override bool CanReduce {

get { return true; }

}

public override Type Type {

get { return typeof(object); }

}

}

Note all this code is untested but should give you the general idea of what to do.

From: asthomas [mailto:notifications@codeplex.com]
Sent: Thursday, July 29, 2010 11:28 AM
To: Dino Viehland
Subject: Re: DLR 1.0 ExpressionType.Assign replaced? [dlr:221868]

From: asthomas

Nope, that didn't help me. Sympl appears to be statically resolving its identifiers. I still have no clue as to how I would construct my assignment expression so that it resolves at execution time instead of at compile time. I understand (somewhat) in principle, but I need a little help on how it could actually be done.

It boils down to this: How can I make or wrap an Expression.Assign() whose LHS is run-time computed? If Expression.Assign cannot do it, what could? I suspect I could make a Lambda expression that will call my own function at runtime, but I'm not sure if that's overkill or exactly how to specify such a thing.

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Jul 29, 2010 at 7:34 PM
That's a lot to absorb! I really appreciate the complete explanation and the concrete guidance, Dino. I think I'll spend some time with this for a while now before I have to beg your help again. That code certainly looks like what I needed. Thanks very much.
Jul 29, 2010 at 10:04 PM

For kicks, I implemented a version of your example. It works, but no claims are made for correctness.

    class Program
    {
        public class MyGetMemberBinder : GetMemberBinder
        {
            public MyGetMemberBinder(string name)
                : base(name, ignoreCase: false)
            {
            }
            public override DynamicMetaObject FallbackGetMember(DynamicMetaObject target, DynamicMetaObject errorSuggestion)
            {
                // Only works on dynamic objects
                return new DynamicMetaObject(
                    Expression.Throw(Expression.New(typeof(NotImplementedException).GetConstructor(Type.EmptyTypes)), target.RuntimeType),
                    BindingRestrictions.Empty
                    );
            }
        }

        public class MySetMemberBinder : SetMemberBinder
        {
            public MySetMemberBinder(string name)
                : base(name, ignoreCase: false)
            {
            }
            public override DynamicMetaObject FallbackSetMember(DynamicMetaObject target, DynamicMetaObject value, DynamicMetaObject errorSuggestion)
            {
                // Only works on dynamic objects
                return new DynamicMetaObject(
                    Expression.Throw(Expression.New(typeof(NotImplementedException).GetConstructor(Type.EmptyTypes)), target.RuntimeType),
                    BindingRestrictions.Empty
                    );
            }
        }

        static Action<object, string, object> MakeAssignmentFunction() {
            var xExpr = Expression.Parameter(typeof(object), "x");
            var nExpr = Expression.Parameter(typeof(string), "n");
            var valueExpr = Expression.Parameter(typeof(object), "value");

            var yExpr = Expression.Dynamic(new MyGetMemberBinder("y"), typeof(object), xExpr);
            var parmnExpr = Expression.Call(null, typeof(string).GetMethod("Concat", new[] { typeof(string), typeof(string) }),
                Expression.Constant("parm"), nExpr);

            // This binder and call site should be cached instead of newing them up each time
            var callSiteType = typeof(CallSite<Func<CallSite, object, object>>);
            var binderExpr = Expression.New(typeof(MyGetMemberBinder).GetConstructor(new[] { typeof(string) }), parmnExpr);
            var callSiteExpr = Expression.Call(null, callSiteType.GetMethod("Create", new[] { typeof(CallSiteBinder) }), binderExpr);
            var callSite = Expression.Variable(callSiteType);
            var callSiteAssign = Expression.Assign(callSite, callSiteExpr);
            var callSiteInvoke = Expression.Invoke(Expression.Field(callSite, "Target"), callSite, yExpr);
            var setZ = Expression.Dynamic(new MySetMemberBinder("z"), typeof(void), callSiteInvoke, valueExpr);
            var block = Expression.Block(new[] { callSite }, callSiteAssign, callSiteInvoke, setZ);
            var lambda = Expression.Lambda(block, new[] { xExpr, nExpr, valueExpr });

            return (Action<object, string, object>)lambda.Compile();
        }

        static void Main(string[] args)
        {
            dynamic x = new ExpandoObject();
            x.y = new ExpandoObject();
            x.y.parm1 = new ExpandoObject();

            var action = MakeAssignmentFunction();

            action(x, "1", 5);

            System.Console.WriteLine(x.y.parm1.z);
        }