Request : better arithmetic support and reusable LambdaCompiler

Mar 21, 2009 at 11:04 PM
Hello

I am in search of a more convenient solution than Emit to build dynamic methods. Unfortunately, the existing expression trees had too much limitations and extending them through heavy hacking did not seem to be a realistic solution. Therefore, I gave a look at the DLR to see how the expression trees were going to be improved. Unfortunately, some of the problems I encountered are still here.

* Arithmetic binary nodes require the both operands to have the exact same type. Which means that I first need to wrap one of the operands into a convert expression. Fine but what if I don't know in advance what the two types are (could be UInt16 and UInt64, float and double, etc) ? Then, I need to write a method to determine, given two types, how they must be converted and added. I can live with it but I doubt I will be the only one to encounter this problem and it's truly a boring scenario which requires to dig the MSIL ECMA specifications to be properly dealt with. It would be convenient to have a way to resolve that for MSIL-level arithmetic operations, so that we're just left with the trouble of resolving the method calls.

* Now, there is one more problem with arithmetic binary nodes : pointer arithmetic. For example, what about adding a pointer and an int ? I don't think I can convert the int into a pointer so I don't see how I could perform that. A solution would be to create my own expression node but the LambdaCompiler is an internal sealed class which cannot be extended. Which means I would need to rewrite it from scratch.

* By the way, making the LambdaCompiler an internal sealed class is a disappointing choice. Is there a good reason for that ? Besides the scenario I already mentioned, I would also like to be able to call a method such as LambdaCompiler.Emit(ILGenerator, Expression) to translate an expresion into an IL code and push it onto the ILGenerator stack. It would also allow us to use expression trees to generate dynamic methods with full visibility on a module.


All in all, just tweaking the LambdaCompiler a bit to make it reusable and extensible would allow me to do some work to make it useful. Without that, I fear I am stuck with Emit while 99% of the required work is already in the DLR. Of course, if you could also enhance the arithmetic support, that would spare me and many others quite some work. I'm waiting for you thoughts on the problem.

Cheers.
Coordinator
Mar 22, 2009 at 10:21 PM

Hi, Harmattan.  As for the first request, while it is tempting to loosen our semantics or make them more convenient for your scenarios, we have legacy compatibility issue with Expr Trees v1.  We’ll add an issue internally to look at some extended factory methods that try to compute good conversions for the operands and a good result type.  It is unlikely we will do this sort of feature work for CLR 4.0.

For your second and third requests, you can use the sources on codeplex to modify for your needs.  You don’t have to start from scratch though you may have duplicate code loaded.  We are starting to think about ETs v3 so this is good information to have.

Thanks,

Bill

From: Harmattan [mailto:notifications@codeplex.com]
Sent: Saturday, March 21, 2009 3:04 PM
To: Bill Chiles
Subject: Request : better arithmetic support and reusable LambdaCompiler [dlr:50851]

From: Harmattan

Hello

I am in search of a more convenient solution than Emit to build dynamic methods. Unfortunately, the existing expression trees had too much limitations and extending them through heavy hacking did not seem to be a realistic solution. Therefore, I gave a look at the DLR to see how the expression trees were going to be improved. Unfortunately, some of the problems I encountered are still here.

* Arithmetic binary nodes require the both operands to have the exact same type. Which means that I first need to wrap one of the operands into a convert expression. Fine but what if I don't know in advance what the two types are (could be UInt16 and UInt64, float and double, etc) ? Then, I need to write a method to determine, given two types, how they must be converted and added. I can live with it but I doubt I will be the only one to encounter this problem and it's truly a boring scenario which requires to dig the MSIL ECMA specifications to be properly dealt with. It would be convenient to have a way to resolve that for MSIL-level arithmetic operations, so that we're just left with the trouble of resolving the method calls.

* Now, there is one more problem with arithmetic binary nodes : pointer arithmetic. For example, what about adding a pointer and an int ? I don't think I can convert the int into a pointer so I don't see how I could perform that. A solution would be to create my own expression node but the LambdaCompiler is an internal sealed class which cannot be extended. Which means I would need to rewrite it from scratch.

* By the way, making the LambdaCompiler an internal sealed class is a disappointing choice. Is there a good reason for that ? Besides the scenario I already mentioned, I would also like to be able to call a method such as LambdaCompiler.Emit(ILGenerator, Expression) to translate an expresion into an IL code and push it onto the ILGenerator stack. It would also allow us to use expression trees to generate dynamic methods with full visibility on a module.


All in all, just tweaking the LambdaCompiler a bit to make it reusable and extensible would allow me to do some work to make it useful. Without that, I fear I am stuck with Emit while 99% of the required work is already in the DLR. Of course, if you could also enhance the arithmetic support, that would spare me and many others quite some work. I'm waiting for you thoughts on the problem.

Cheers.

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Mar 23, 2009 at 10:03 AM
Hi, Billchi.

Thanks for your answer. I was expecting deadline and legacy problems but I'm glad to know you could look at those problems for the future. Regarding the possibility to modify the existing version, it may seem strange, but it only came to my mind after my post that the Ms-PL was indeed allowing me to do that. Seems like I had difficulties to realize that I am able to legally reuse and transform a code written by a Ms team but I have to say it's both convenient and a very nice feeling. Finally, assuming the future versions of the DLR will stay under Ms-PL (or that you will bring some of the changes I suggested in the future versions), it will be easier to maintain the code against possible future changes of the CLR by relying on your own work for that.

Congratulations for the great work made so far, the DLR opens us bright new perspectives.