Strange IronPython behavior

Jun 17, 2010 at 11:15 PM
Edited Jun 17, 2010 at 11:15 PM

Toying with the IronPython interpreter I built, I was baffled by the following behavior:

>>> 2 + 2
4
>>> def square(x) : return x ** 2

>>> square(5)
25
>>> sum(square(x) for x in range(10))
global name 'square' is not defined
>>> square
<function square at 0x000000000000002B>
>>> sum(x ** 2 for x in range(10))
285

Any idea why the sum of square(x) may fail? The same code runs perfectly if you try it inside the Silverlight interpreter at www.trypython.org (also based on IronPython and the DLR).

Coordinator
Jun 17, 2010 at 11:20 PM

How are you creating the scopes and the code? the reference to square inside of the call to sum() is actually inside of a generator which includes a new function scope. So I assume there’s something weird w/ the scopes which is causing the lookup to fail but I don’t have a clue off the top of my head what it would be.

From: jdu [mailto:notifications@codeplex.com]
Sent: Thursday, June 17, 2010 4:16 PM
To: Dino Viehland
Subject: Strange IronPython behavior [dlr:216453]

From: jdu

Toying with any IronPython interpreter I built, I was baffled by the following behavior:

>>> 2 + 2
4
>>> def square(x) : return x ** 2
 
>>> square(5)
25
>>> sum(square(x) for x in range(10))
global name 'square' is not defined
>>> square
<function square at 0x000000000000002B>
>>> sum(x ** 2 for x in range(10))
285

Any idea why the sum of square(x) may fail? The same code runs perfectly if you try it inside the Silverlight interpreter at www.trypython.org (also based on IronPython and the DLR).

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Jun 18, 2010 at 1:50 PM

Thank you so much for your insight. I'm not really a Python guy. Your remark about a generator being a new function scope is probably right on spot. My square function works pretty much everywhere else. Although I still don't understand exactly how I'd fix it.

My interpreter works that way:

At constructor time, I create a new PythonEngine and a new PythonScope.

Each time a line is entered I use engine.CreateScriptSourceFromString (with SourceCode.InteractiveCode) to create a ScriptSource and check whether the interpreter should allow for continuation on a new line.

Then when I think the code is complete, I use source.Execute(scope) to try to execute the code. (scope being always the same ScriptScope at each call: the one created in the constructor).

The error saying "global name 'square'" and your remark about generator having a new scope makes me wonder if the square function is defined in my own "scope" object rather than globally and if the generator scope doesn't have access to it. But how should I fix that?

I like being able to define different scopes when executing various pieces of Python script, and I like the possibility to "throw away" a scope and start in a clean environment again. Ideally the fix for my problem should let me do that.

Jun 18, 2010 at 7:05 PM
Edited Jun 18, 2010 at 7:07 PM

After much reading about Python, I'm starting to think it's a bug in IronPython.

Let's play a bit. My problem is, you'd recall:

>>> def square(x) : return x ** 2 
>>> sum(square(x) for x in range(10)) 
global name 'square' is not defined 

But pursuing the investigation, I think it's funny to notice that global name 'square' IS defined:

>>> globals().keys() 
['__builtins__', 'square'] 

And why does this work in the silverlight trypython.org ? Looking at the source code, their interpreter is written in Pyhton itself and calls exec. So let's try that:

>>> exec 'print sum(square(x) for x in range(10))' 
285 

Now I'm pretty sure there's something wrong in the way ScriptSource.Execute works... :-(
Any idea?

Coordinator
Jun 18, 2010 at 7:52 PM

You’re right that this definitely looks like a bug w/ IronPython – I also think it’s likely a regression introduced in 2.6.1 from 2.6.0.

The problem is that we are doing a re-write of the tree to de-optimize global variable access in this case and it looks like we don’t handle when generators are present. Therefore we’re actually looking for square in the wrong place.

I think I have a fix – it certainly fixes the issue I just need to make sure it doesn’t break anything else. We should be able to put the fix in 2.6.2 but for the time being you could try going back to 2.6.0 to work around the issue. Or if you’re willing to build IronPython yourself you can add:

GeneratorExpression generator = node as GeneratorExpression;

if (generator != null) {

return base.VisitExtension(new GeneratorExpression((FunctionDefinition)VisitScope(generator.Function), generator.Iterable));

}

Into PythonAst.cs right after:

LambdaExpression lambda = node as LambdaExpression;

if (lambda != null) {

return base.VisitExtension(new LambdaExpression((FunctionDefinition)VisitScope(lambda.Function)));

}

From: jdu [mailto:notifications@codeplex.com]
Sent: Friday, June 18, 2010 12:06 PM
To: Dino Viehland
Subject: Re: Strange IronPython behavior [dlr:216453]

From: jdu

After much reading about Python, I'm starting to think it's a bug in IronPython. Let's play a bit. My problem is, you'd recall: >>> def square(x) : return x ** 2 >>> sum(square(x) for x in range(10)) global name 'square' is not defined But pursuing the investigation, I think it's funny to notice that global name 'square' IS defined: >>> globals().keys() ['__builtins__', 'square'] And why does this work in the silverlight trypython.org ? Looking at the source code, their interpreter is written in Pyhton itself and calls exec. So let's try that: >>> exec 'print sum(square(x) for x in range(10))' 285 Now I'm pretty sure there's something wrong in the way ScriptSource.Execute works... :-(

Read the full discussion online.

To add a post to this discussion, reply to this email (dlr@discussions.codeplex.com)

To start a new discussion for this project, email dlr@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Jun 18, 2010 at 9:05 PM

Thank you very much for your quick answers!

FYI I've applied your 3-liners patch and it works.

Having Python integrated inside my application is awesome, I can't stop coming up with new cool possibilities! The scripting stuff is really an excellent addition to the .NET framework.