classification
Title: UnboundLocalError when trying to raise exceptions inside execfile
Type: behavior Stage:
Components: Interpreter Core Versions: Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: amaury.forgeotdarc Nosy List: amaury.forgeotdarc, jerry.seutter, jhylton, nikolasco, tenuki, werneck
Priority: normal Keywords: patch

Created on 2008-03-18 02:55 by jerry.seutter, last changed 2008-07-21 22:01 by amaury.forgeotdarc. This issue is now closed.

Files
File name Uploaded Description Edit
core_wierd_bug_minimal.zip jerry.seutter, 2008-03-18 02:55 Files listed in comments above, in a zipfile. "figleaf" and "test_broken.py"
test_broken3.py tenuki, 2008-06-22 02:38
debughelper.tgz tenuki, 2008-06-22 02:59
localstofast.patch amaury.forgeotdarc, 2008-07-21 09:23
Messages (10)
msg63856 - (view) Author: Jerry Seutter (jerry.seutter) * (Python committer) Date: 2008-03-18 02:55
Found a bug when trying to integrate figleaf coverage into trunk.  I
have ripped the code down to the smallest subset that still causes the
behaviour.  The code works on the latest release of Python 2.5 but is
broken on trunk.  It comes in two files.  The first is the caller (figleaf):

import os
import sys

def foo(f, e, o):
    pass
    
sys.settrace(foo)

import __main__
execfile('test_broken.py', __main__.__dict__)



The second file is the test (test_broken.py):

# This code breaks on trunk

def test_foo():
    class CustomException(Exception):
        pass

    class SomeClass:
        def foo(self):
            raise CustomException

    # The error only appears with enough nested blocks.
    if (True == True):
        try:
            raise IOError
        except CustomException:
            pass



It should raise IOError.  When run, it gives the following output:

jerry-seutters-computer:~/code/python/core_wierd_bug_minimal jseutter$
../core/python.exe figleaf                
Traceback (most recent call last):
  File "figleaf", line 10, in <module>
    execfile('test_broken.py', __main__.__dict__)
  File "test_broken.py", line 18, in <module>
    test_foo()
  File "test_broken.py", line 15, in test_foo
    except CustomException:
UnboundLocalError: local variable 'CustomException' referenced before
assignment
[10019 refs]
msg66575 - (view) Author: Pedro Werneck (werneck) Date: 2008-05-10 19:54
Just note the error happens even without the try/except block inside the
'if' statement.
msg66662 - (view) Author: Nikolas Coukouma (nikolasco) Date: 2008-05-11 20:21
I can't reproduce this with r63075...
msg66665 - (view) Author: Pedro Werneck (werneck) Date: 2008-05-11 20:45
I get it with r63075, r63085, on Linux.
msg66674 - (view) Author: Nikolas Coukouma (nikolasco) Date: 2008-05-11 21:34
Apologies, I didn't run the test case correctly; I do get the error as
reported
msg68554 - (view) Author: alejandro david weil (tenuki) Date: 2008-06-22 02:38
Shorter trigger code..
msg68555 - (view) Author: alejandro david weil (tenuki) Date: 2008-06-22 02:59
Some debugging helper code and my conclutions of one work day:
debughelper.tgZ:
-test_broken1/2.py  
       one does triggers the bug, the other doesn't)
-rtest.sh 
       executes boths and compares its outputs
-frameobject.c.diff
       applied to Objects/frameobject.c, adds some debug info.

What I found:
1. The CustomException is disappearing from locals()
2. PyFrame_FastToLocals() (from that .c file) is updating the locals, 
and removing that exception from there.
3. In the failing case this code:
                if (deref) {
                        assert(PyCell_Check(value));
                        value = PyCell_GET(value);
                }
 is returning value==NULL.

Don't know why that happens.
But you could inspect out1.txt/out2.txt made with rtest.sh, and could 
discover something..
msg68613 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-06-23 07:34
The problem seems to have been introduced by r53954.
msg68617 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-06-23 08:54
The problem is in PyFrame_LocalsToFast.

(As I understand it: "Locals" refers to the locals() dictionary of the
frame; "Fast" refers to an optimization where local variables are stored
in an array. 
Each call to locals() or the trace function normalizes data by copying
the Fast array into the Locals dictionary; with sys.settrace, the user
may modify the locals, so the inverse transformation is done after each
call to the trace function)

When defining a class, PyFrame_FastToLocals does not copy the free
variables from enclosing scopes. This is the goal of r53954 "Do not copy
free variables to locals in class namespaces".
When executing code in the class body, the sys.settrace function is
called with this reduced set of locals().
The problem is that PyFrame_LocalsToFastdoes does not have the same
test, and uses the locals() to update (and clear) the free variables as
well.

I attach a patch that makes PyFrame_LocalsToFast symmetric to
PyFrame_FastToLocals: in the case of a class statement, do not update 
the free variables.
msg70122 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-07-21 22:01
Committed as r65177.
History
Date User Action Args
2008-07-21 22:01:19amaury.forgeotdarcsetstatus: open -> closed
resolution: fixed
messages: + msg70122
2008-07-21 09:29:43amaury.forgeotdarclinkissue3415 superseder
2008-07-21 09:27:05amaury.forgeotdarcsetassignee: amaury.forgeotdarc
2008-07-21 09:24:04amaury.forgeotdarcsetfiles: - localstofast.patch
2008-07-21 09:23:56amaury.forgeotdarcsetfiles: + localstofast.patch
2008-06-23 08:54:33amaury.forgeotdarcsetfiles: + localstofast.patch
nosy: + jhylton
messages: + msg68617
keywords: + patch
2008-06-23 07:34:11amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg68613
2008-06-22 02:59:52tenukisetfiles: + debughelper.tgz
messages: + msg68555
2008-06-22 02:38:15tenukisetfiles: + test_broken3.py
nosy: + tenuki
messages: + msg68554
2008-05-11 21:34:01nikolascosetmessages: + msg66674
2008-05-11 20:45:39wernecksetmessages: + msg66665
2008-05-11 20:21:47nikolascosetnosy: + nikolasco
messages: + msg66662
2008-05-10 19:54:25wernecksetnosy: + werneck
messages: + msg66575
2008-03-18 02:56:49jerry.seuttersetcomponents: + Interpreter Core
2008-03-18 02:55:44jerry.seuttercreate