Deadlock while loading assemblies/Executing, need work around ideas.

May 13, 2011 at 8:48 PM

We run embedded IronPython (2.0.1) in our application in order to provide an extensibility point for our end users. We provide one static runtime/engine for the app to run user scripts against. We load a small set of our assemblies into the runtime at app startup, but then load a bunch more assemblies into the runtime as they load into our apps appdomain. A problem we sporadically run into is deadlock somewhere deep in the engine/runtime because we are loading an assembly into the runtime on a background thread at the same time a script is executing on the main UI thread. Unfortunately, I don't have a call stack for you at this time. 
This is a tough situation for us to work around. Our app takes full advantage of multi-threading and at any time we could be loading assemblies into the runtime on the background thread and executing on the main thread or vise versa. Having witnessed the deadlock we need to come up with a better solution for loading assemblies into our default runtime. Is there perhaps a mechanism that fires an event if the engine can't find an assembly during execution? Something we could capture and load at that time? Any other advice?
Our static default script runtime, loaded with our assemblies, is one of the large memory consumers we have in our product. I am pretty sure that that is why we chose to not implement separate runtimes for each script executed and/or maybe the performance hit for creating them. The developers for this part of our app are all gone. If it is a catastrophic design flaw to run a static runtime/engine and thread safety is not guaranteed please let me know. Any help and/or ideas would be very appreciated. Thanks.

Coordinator
May 13, 2011 at 8:54 PM

You can hook the assembly resolve event on the current app domain - System.AppDomain.CurrentDomain.AssemblyResolve. You’ll then get called whenever the CLR fails to load an assembly.

It’s not a catastrophic design flow to have a single ScriptRuntime. The users scripts could possibly interfere with each other (e.g. by modifying shared modules) but the IronPython runtime is designed to be thread safe so modulo any bugs this should work just fine.

If you can get the stack trace of the deadlock it’d help determine if this is an issue w/ IronPython or with other code (for example it could be two static constructors deadlocking, or something completely different). Unfortunately 2.0 is really old and no one is actively maintaining it. But if the issue was obvious I might be able to point you at a solution if you were to re-build IronPython yourselves.

From: Blitz357 [email removed]
Sent: Friday, May 13, 2011 1:49 PM
To: Dino Viehland
Subject: Deadlock while loading assemblies/Executing, need work around ideas. [dlr:257585]

From: Blitz357

We run embedded IronPython (2.0.1) in our application in order to provide an extensibility point for our end users. We provide one static runtime/engine for the app to run user scripts against. We load a small set of our assemblies into the runtime at app startup, but then load a bunch more assemblies into the runtime as they load into our apps appdomain. A problem we sporadically run into is deadlock somewhere deep in the engine/runtime because we are loading an assembly into the runtime on a background thread at the same time a script is executing on the main UI thread. Unfortunately, I don't have a call stack for you at this time.
This is a tough situation for us to work around. Our app takes full advantage of multi-threading and at any time we could be loading assemblies into the runtime on the background thread and executing on the main thread or vise versa. Having witnessed the deadlock we need to come up with a better solution for loading assemblies into our default runtime. Is there perhaps a mechanism that fires an event if the engine can't find an assembly during execution? Something we could capture and load at that time? Any other advice?
Our static default script runtime, loaded with our assemblies, is one of the large memory consumers we have in our product. I am pretty sure that that is why we chose to not implement separate runtimes for each script executed and/or maybe the performance hit for creating them. The developers for this part of our app are all gone. If it is a catastrophic design flaw to run a static runtime/engine and thread safety is not guaranteed please let me know. Any help and/or ideas would be very appreciated. Thanks.