Understanding how the BinaryOperationBinder works

May 11, 2012 at 8:46 AM

Hello DLR architects,

I wish to understand how the BinaryOperationBinder works. For this I wrote a minimalistic test language, see [1], that try to execute the following snippet: "return 'abc' < 'xyz'" Using the test language, I am trying to get a boolean result from a engine.Execute(code) call, were engine is the ScriptEngine of my language.

In the CompileSourceCode() function of my language context, I create the lambda expression which will be eventually executed, as follows:

public override ScriptCode CompileSourceCode(..)
{
  var left = Expression.Constant("abc");
  var right = Expression.Constant("xyz");
  var opr = ExpressionType.LessThan;

  var binExpr = Expression.Dynamic(
        context.CreateBinaryOperationBinder(opr),
        typeof(bool), left, right);
       
  var binExprObj = Expression.Convert(binExpr, typeof(object));
 
  var statements = new List<Expression>();
 
  var blockReturnLabel = Expression.Label(typeof(object));
  statements.Add(Expression.Return(blockReturnLabel, binExprObj, typeof(object)));

  statements.Add(Expression.Label(blockReturnLabel, Expression.Constant(null)));

  var block = Expression.Block(typeof(object), statements);
  var code = Expression.Lambda<Func<dynamic>>(block); 

  Func<dynamic> compiledCode = code.Compile();
  return new MyScriptCode(sourceUnit, compiledCode); 
}

Now my understanding is, that this should work. But this is obviously wrong because executing the code throws an exception, complaining about type System.Object of the binder not being compatible with the type expected at the call site. And I don't understand why. What is missing?

Can someone shed some light on why?

Thanks,
  Francois
 
[1] https://github.com/fgretief/IronLua/blob/master/Sample/MyLanguage.cs

May 16, 2012 at 4:28 PM

Hello DLRs,

I eventually found the answer to the problem. Actually there were two issues.

The first was that the ReturnType of a BinaryOperationBinder is Object.
And my code told the Expression.Dynamic method to return binary. So replacing
the typeof(bool) with typeof(object) fixed that issue.

  var binExpr = Expression.Dynamic(
        context.CreateBinaryOperationBinder(opr),
        typeof(object), left, right);

The second issue was again with the BinaryOperationBinder returning an Object.
Inside the FallbackBinaryOperation method of my binder, I made a call to the
DefaultBinder's DoOperation() method, which return a boolean type. This was the
reason why I added boolean to the Expression.Dynamic call in the first place.
So we need to convert the output to the return type of the binder.

    public override DynamicMetaObject FallbackBinaryOperation(
        DynamicMetaObject target,
        DynamicMetaObject arg,
        DynamicMetaObject errorSuggestion)
    {
        ...        
        
        // original code
        //return _binder.DoOperation(Operation, left, right);
        
        // new code
        var returnType = this.ReturnType;
        var mo = context.Binder.DoOperation(Operation, left, right);
        if (mo.Expression.Type != returnType)
        {
            mo = mo.Clone(Expression.Convert(mo.Expression, returnType));
        }
        return mo;
    }

Now my example executes as expected.
    
Regards,
  Francois

Hello DLRs,

I eventually found the answer to the problem. Actually there were two issues.

The first was that the ReturnType of a BinaryOperationBinder is Object.
And my code told the Expression.Dynamic method to return binary. So replacing
the typeof(bool) with typeof(object) fixed that issue.

  var binExpr = Expression.Dynamic(
        context.CreateBinaryOperationBinder(opr),
        typeof(object), left, right);

The second issue was again with the BinaryOperationBinder returning an Object.
Inside the FallbackBinaryOperation method of my binder, I made a call to the
DefaultBinder's DoOperation() method, which return a boolean type. This was the
reason why I added boolean to the Expression.Dynamic call in the first place.
So we need to convert the output to the return type of the binder.

    public override DynamicMetaObject FallbackBinaryOperation(
        DynamicMetaObject target,
        DynamicMetaObject arg,
        DynamicMetaObject errorSuggestion)
    {
        ...       
       
        // original code
        //return _binder.DoOperation(Operation, left, right);
       
        // new code
        var returnType = this.ReturnType;
        var mo = context.Binder.DoOperation(Operation, left, right);
        if (mo.Expression.Type != returnType)
        {
            mo = mo.Clone(Expression.Convert(mo.Expression, returnType));
        }
        return mo;
    }
   
Regards,
  Francois
Coordinator
May 16, 2012 at 4:32 PM

Thanks for the followup, and very sorry it took us a while to get back to you. I just got some insight from dev this morning since I was not sure of the answer. I won't include his text on the change you sited, but here's a bit of extra info that may be interesting.

Almost all of the standard dynamic operations are expected to return object – in this particular case the problem is less than can be overloaded to return non-bool values (even in static languages like C#). Therefore the DLR requires that you always return object. Some of the logic behind this is probably expressed in these stack overflow posts:

http://stackoverflow.com/questions/1360097/dlr-return-type

http://stackoverflow.com/questions/1835912/how-do-i-express-a-void-method-call-as-the-result-of-dynamicmetaobject-bindinvok/1836591#1836591

So this needs to be declared to return object, and then if you really want a bool you need to add a dynamic convert to bool site – but it doesn’t look like that’s needed here. Languages can be smarter and can implement their own “Less than + convert to bool sites” which combine the 2 operations into a single operation. The DLR outer layer even includes the ComboBinder to automatically combine the 2 dynamic operations into a single one given sub-binders which work correctly. But the core operations are always fixed to returning object (or void)

Bill

From: fgretief [email removed]
Sent: Wednesday, May 16, 2012 9:29 AM
To: Bill Chiles
Subject: Re: Understanding how the BinaryOperationBinder works [dlr:355421]

From: fgretief

Hello DLRs,

I eventually found the answer to the problem. Actually there were two issues.

The first was that the ReturnType of a BinaryOperationBinder is Object.
And my code told the Expression.Dynamic method to return binary. So replacing
the typeof(bool) with typeof(object) fixed that issue.

var binExpr = Expression.Dynamic(
context.CreateBinaryOperationBinder(opr),
typeof(object), left, right);

The second issue was again with the BinaryOperationBinder returning an Object.
Inside the FallbackBinaryOperation method of my binder, I made a call to the
DefaultBinder's DoOperation() method, which return a boolean type. This was the
reason why I added boolean to the Expression.Dynamic call in the first place.
So we need to convert the output to the return type of the binder.

public override DynamicMetaObject FallbackBinaryOperation(
DynamicMetaObject target,
DynamicMetaObject arg,
DynamicMetaObject errorSuggestion)
{
...

// original code
//return _binder.DoOperation(Operation, left, right);

// new code
var returnType = this.ReturnType;
var mo = context.Binder.DoOperation(Operation, left, right);
if (mo.Expression.Type != returnType)
{
mo = mo.Clone(Expression.Convert(mo.Expression, returnType));
}
return mo;
}

Now my example executes as expected.

Regards,
Francois

Hello DLRs,

I eventually found the answer to the problem. Actually there were two issues.

The first was that the ReturnType of a BinaryOperationBinder is Object.
And my code told the Expression.Dynamic method to return binary. So replacing
the typeof(bool) with typeof(object) fixed that issue.

var binExpr = Expression.Dynamic(
context.CreateBinaryOperationBinder(opr),
typeof(object), left, right);

The second issue was again with the BinaryOperationBinder returning an Object.
Inside the FallbackBinaryOperation method of my binder, I made a call to the
DefaultBinder's DoOperation() method, which return a boolean type. This was the
reason why I added boolean to the Expression.Dynamic call in the first place.
So we need to convert the output to the return type of the binder.

public override DynamicMetaObject FallbackBinaryOperation(
DynamicMetaObject target,
DynamicMetaObject arg,
DynamicMetaObject errorSuggestion)
{
...

// original code
//return _binder.DoOperation(Operation, left, right);

// new code
var returnType = this.ReturnType;
var mo = context.Binder.DoOperation(Operation, left, right);
if (mo.Expression.Type != returnType)
{
mo = mo.Clone(Expression.Convert(mo.Expression, returnType));
}
return mo;
}

Regards,
Francois

May 30, 2012 at 7:27 PM
billchi wrote:


Languages can be smarter and can implement their own “Less than + convert to bool sites” which combine the 2 operations into a single operation. The DLR outer layer even includes the ComboBinder to automatically combine the 2 dynamic operations into a single one given sub-binders which work correctly.

Can you provide a small concrete example of this? I'm currently implementing a language, and doing stuff like "if(a < b)" turns out to be a pain because of stuff like this, but even more, I've been completely unable to use

Utils.LightDynamic(engine.GetUnaryBinder(ExpressionType.IsTrue), valueExpr)

because it complains about the binder returning a bool while the callsite requires an object-result, and this is before any of my code even runs! So what seems to be the problem is that I should probably specify the call-site type-argument myself, but I've found myself unable to figure out how to do that from Expression-objects. Even conversion to bool requires an object-result because the call-site is miss-matched.

What I currently do is parse my language, then hand it generate an Expression-tree, and hand a LightLambda of that delegate over to a SourceCode-object. How can I change the generation of the Expression-tree to enable stuff like this?