Implementing new language using the DLR

Jul 20, 2009 at 3:03 PM

Hi,

I just skimmed through the source code and figured that ruby and python do not actually use the DLR expressions, but implement their own.

When trying to implement a new language, should I use my own Expressions or should I use the DLR tree ?

Coordinator
Jul 20, 2009 at 4:11 PM

Actually what you’re seeing there is that most languages first parse to their own abstract syntax tree, then as a distinct phase or as part of semantic analysis, they later convert to Expression Trees before compiling.  They also use ETs in the DLR’s dynamic call site caching mechanism, as the “rules” for how to perform a given call site’s operation on the current objects flowing into the site is captured as ETs.

You should check out the Sympl language sample on the codeplex site.  It has full source code in python and C#, and there’s a document that walks you through the language implementation in terms of how to use the dynamic call site caching, ETs, interop, very very fundamental .NET binding, etc.  The language implements several features, including supporting binding to globals from a host and implementing modules.  Sympl also first parses to its own AST, then to ETs to compile and run code.

Cheers,

Bill

From: Krasshirsch [mailto:notifications@codeplex.com]
Sent: Monday, July 20, 2009 7:03 AM
To: Bill Chiles
Subject: Implementing new language using the DLR [dlr:62963]

From: Krasshirsch

Hi,

I just skimmed through the source code and figured that ruby and python do not actually use the DLR expressions, but implement their own.

When trying to implement a new language, should I use my own Expressions or should I use the DLR tree ?

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Jul 20, 2009 at 4:22 PM

I would suggest you to create your own abstract syntax tree (AST) that closely model your language. You can then walk your tree to generate the DLR tree and have the DLR compile the DLR tree. With your own tree, you can model the semantics of your language better so that it is easier and more accurate to catch and report syntax errors.

Jul 20, 2009 at 7:36 PM

Thanks for both of your replies.

I just followed your recommendation and read through the Sympl example code, unfortunately it is still a little chaotic for my eyes, I'll need some time to digest it ;)

 

On this note I'd like to ask a more general question concerning the DLR.

I'd like to extend a .NET application by adding a scripting layer, to be able to customize it for future needs, similar to the VSTA environment that you can embed into exising applications.

VSTA does not use the DLR just compiling and loading assemblies at runtime. Is it possible to do all the things using the DLR that can be done by the CLR or is trying to mimic VSTA using the DLR a futile attempt ?

 

 

Coordinator
Jul 20, 2009 at 9:26 PM

I think if you at least skim the doc, it will orient you and give you some site lines through the code.

Your questions are interesting.  Let me make some statements and see if I narrow the field a bit for you.  The DLR lowers the bar for implementing a dynamic language on .NET while also providing mechanisms for language interop and common hosting API.  The hosting APIs go only as far as code execution (create ScriptRuntime, store host globals, get a language engine, execute code, pull out some resulting objects perhaps, and so on).  VSTA provides a full stack of APIs for managing projects of VB.NET code, macro recording host actions as VB.NET source code against the host’s object model, tooling, debugging UI, code execution, etc.  When using the DLR, you have the entire CLR at your disposal; however, the DLR expr trees are not as powerful as emitting MSIL directly yourself into an assembly (for example, unsafe code and other full IL power).

You might check out the small DLR hosting sample in the ...\runtime\samples\ directory that Sesh wrote.  It is a little drawing app with a few basic shapes, an object model, hosting the DLR for manipulating the drawing surface, and a little REPL for doing so interactively.  I think his blog (http://blogs.msdn.com/seshadripv/ ) walks you through it.  The DLR hosting spec shows some snippets and explains the model with the API refs.

Bill

From: Krasshirsch [mailto:notifications@codeplex.com]
Sent: Monday, July 20, 2009 11:37 AM
To: Bill Chiles
Subject: Re: Implementing new language using the DLR [dlr:62963]

From: Krasshirsch

Thanks for both of your replies.

I just followed your recommendation and read through the Sympl example code, unfortunately it is still a little chaotic for my eyes, I'll need some time to digest it ;)

On this note I'd like to ask a more general question concerning the DLR.

I'd like to extend a .NET application by adding a scripting layer, to be able to customize it for future needs, similar to the VSTA environment that you can embed into exising applications.

VSTA does not use the DLR just compiling and loading assemblies at runtime. Is it possible to do all the things using the DLR that can be done by the CLR or is trying to mimic VSTA using the DLR a futile attempt ?

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Jul 20, 2009 at 10:01 PM

I only raised this comparison, because I believe VSTA to be a very powerful tool, when it comes to creating extensible applications, unfortunately it has some drawbacks which make it unusable in our application.

VSTA is only suitable for desktop applications, since it can only output Winforms and WPF forms, it's just not ment to work on the server side, I doubt there is even a licencing model for this kind of usage.

Our application is accessed through the web using html/silverlight. 

We hoped to be able to recreate some features from VSTA using the Visual Studio Shell and the DLR on the server side.  

The DLR appearently uses caches to avoid redundant compilation, so speed should not be a problem, since the code won't be altered often.

I'm just a little concerned about the capabilities. Evaluating a couple of expressions and popping out a result is one thing, declaring complex structures is another, or am I wrong here ?

Let's assume we manage to implement C# on top of the DLR, cutting out all those "can't do's" like unsafe code, will I be able to create classes, interfaces at runtime or will I need to use the CompilerServices embedded into the Framework for this ?

 

 

 

Coordinator
Jul 20, 2009 at 10:14 PM

Since you’re more interested in server side or RIA, maybe Jimmy will jump in here (he’s out this week or part of it though).  Yes, if you want to be able to define classes and so on, you need to go below the DLR, like IPy does.  IPy and IRuby do their own ref.emit / CLR generation of classes and types in dynamic assemblies, then emit DLR ETs to run code against them or gen code that they store as members in these classes.

Bill

From: Krasshirsch [mailto:notifications@codeplex.com]
Sent: Monday, July 20, 2009 2:02 PM
To: Bill Chiles
Subject: Re: Implementing new language using the DLR [dlr:62963]

From: Krasshirsch

I only raised this comparison, because I believe VSTA to be a very powerful tool, when it comes to creating extensible applications, unfortunately it has some drawbacks which make it unusable in our application.

VSTA is only suitable for desktop applications, since it can only output Winforms and WPF forms, it's just not ment to work on the server side, I doubt there is even a licencing model for this kind of usage.

Our application is accessed through the web using html/silverlight.

We hoped to be able to recreate some features from VSTA using the Visual Studio Shell and the DLR on the server side.

The DLR appearently uses caches to avoid redundant compilation, so speed should not be a problem, since the code won't be altered often.

I'm just a little concerned about the capabilities. Evaluating a couple of expressions and popping out a result is one thing, declaring complex structures is another, or am I wrong here ?

Let's assume we manage to implement C# on top of the DLR, cutting out all those "can't do's" like unsafe code, will I be able to create classes, interfaces at runtime or will I need to use the CompilerServices embedded into the Framework for this ?

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Jul 20, 2009 at 11:41 PM

Krasshirsch, the DLR itself runs in Silverlight as well. While implementing your language on the DLR you need to make sure it works in Silverlight as well.

You can build the latest sources for Silverlight like so:

msbuild CodeplexDLR.sln /p:Configuration="Silverlight Debug" /p:SilverlightSdk="C:\Program Files\Microsoft Silverlight\3.0.40624.0"

~js

From: billchi [mailto:notifications@codeplex.com]
Sent: Monday, July 20, 2009 2:14 PM
To: Jimmy Schementi
Subject: Re: Implementing new language using the DLR [dlr:62963]

From: billchi

Since you’re more interested in server side or RIA, maybe Jimmy will jump in here (he’s out this week or part of it though). Yes, if you want to be able to define classes and so on, you need to go below the DLR, like IPy does. IPy and IRuby do their own ref.emit / CLR generation of classes and types in dynamic assemblies, then emit DLR ETs to run code against them or gen code that they store as members in these classes.

Bill

From: Krasshirsch [mailto:notifications@codeplex.com]
Sent: Monday, July 20, 2009 2:02 PM
To: Bill Chiles
Subject: Re: Implementing new language using the DLR [dlr:62963]

From: Krasshirsch

I only raised this comparison, because I believe VSTA to be a very powerful tool, when it comes to creating extensible applications, unfortunately it has some drawbacks which make it unusable in our application.

VSTA is only suitable for desktop applications, since it can only output Winforms and WPF forms, it's just not ment to work on the server side, I doubt there is even a licencing model for this kind of usage.

Our application is accessed through the web using html/silverlight.

We hoped to be able to recreate some features from VSTA using the Visual Studio Shell and the DLR on the server side.

The DLR appearently uses caches to avoid redundant compilation, so speed should not be a problem, since the code won't be altered often.

I'm just a little concerned about the capabilities. Evaluating a couple of expressions and popping out a result is one thing, declaring complex structures is another, or am I wrong here ?

Let's assume we manage to implement C# on top of the DLR, cutting out all those "can't do's" like unsafe code, will I be able to create classes, interfaces at runtime or will I need to use the CompilerServices embedded into the Framework for this ?

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Jul 21, 2009 at 12:12 AM

Krasshirsch, if you are interested in server-side hosting of DLR, I wrote a small sample a few months back while playing with DLR: http://www.dotneteer.com/Weblog/post/2009/04/Announcing-Toyscript-Server-Pages-(TSP).aspx. It used the Toyscript sample that has been discontinued since 0.91 release. I guess I could update the sample to use Simpl as Simpl now supports hosting API when I get some time. The sample hosts DLR in an ASP.NET HttpHandler and executes scripts embedded in a classic asp-like page. It is fairly simple, but does show how to host DLR and expose objects from hosting applicatiion to DLR.