classification
Title: SystemError when method has both super() & closure
Type: behavior Stage: commit review
Components: Interpreter Core Versions: Python 3.0
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: barry Nosy List: amaury.forgeotdarc, barry, brett.cannon, christian.heimes, kaizhu
Priority: release blocker Keywords: patch

Created on 2008-11-20 10:29 by kaizhu, last changed 2008-11-20 20:02 by barry. This issue is now closed.

Files
File name Uploaded Description Edit
super-withcell.patch amaury.forgeotdarc, 2008-11-20 14:16
Messages (8)
msg76093 - (view) Author: kai zhu (kaizhu) Date: 2008-11-20 10:29
################################
# super_closure.py
class A(object):
  def foo(self):
    return super()
    # remove the closure below
    # & SystemError goes away ???
    lambda: self
A().foo()
################################

when run on 3.0rc1 & 3.0rc2:

hpc-login2 3 ~/work/py3to2: python3.0 super_closure.py
Traceback (most recent call last):
  File "super_closure.py", line 9, in <module>
    A().foo()
  File "super_closure.py", line 5, in foo
    return super()
SystemError: super(): __class__ is not a type (A)

SystemError seems to b raised from typeobject.c (line6155):

static int
super_init(PyObject *self, PyObject *args, PyObject *kwds)
{...
        if (!PyType_Check(type)) {
            PyErr_Format(PyExc_SystemError,
              "super(): __class__ is not a type (%s)",
              Py_TYPE(type)->tp_name);
            return -1;
        }
        break;
msg76094 - (view) Author: kai zhu (kaizhu) Date: 2008-11-20 10:59
here's a printout of bytecode from script

>>>
>>> s = open("super_closure.py").read()
>>> c = compile(s, "super_closure.py", "exec")
>>> t = py3to2.codetree(c)
>>> print( t )
codetree(
co_argcount =     0,
co_cellvars =     (),
co_code =        
b'Gd\x00\x00\x84\x00\x00d\x01\x00e\x00\x00\x83\x03\x00Z\x01\x00e\x01\x00\x83\x00\x00j\x02\x00\x83\x00\x00\x01d\x02\x00S',
co_filename =     'super_closure.py',
co_firstlineno =  3,
co_flags =        64,
co_freevars =     (),
co_kwonlyargcount =0,
co_lnotab =       b'\x13\x06',
co_name =         '<module>',
co_names =        ('object', 'A', 'foo'),
co_nlocals =      0,
co_stacksize =    4,
co_varnames =     (),
depth =           0,
co_consts = (
 codetree(
 co_argcount =     1,
 co_cellvars =     ('__class__',),
 co_code =        
b'|\x00\x00Ee\x00\x00Z\x01\x00\x87\x00\x00f\x01\x00d\x00\x00\x86\x00\x00Z\x02\x00\x87\x00\x00S',
 co_filename =     'super_closure.py',
 co_firstlineno =  3,
 co_flags =        2,
 co_freevars =     (),
 co_kwonlyargcount =0,
 co_lnotab =       b'\n\x01',
 co_name =         'A',
 co_names =        ('__name__', '__module__', 'foo'),
 co_nlocals =      1,
 co_stacksize =    2,
 co_varnames =     ('__locals__',),
 depth =           1,
 co_consts = (
  codetree(
  co_argcount =     1,
  co_cellvars =     ('self',),
  co_code =        
b't\x00\x00\x83\x00\x00S\x87\x00\x00f\x01\x00d\x01\x00\x86\x00\x00\x01',
  co_filename =     'super_closure.py',
  co_firstlineno =  4,
  co_flags =        3,
  co_freevars =     ('__class__',),
  co_kwonlyargcount =0,
  co_lnotab =       b'\x00\x01\x07\x03',
  co_name =         'foo',
  co_names =        ('super',),
  co_nlocals =      1,
  co_stacksize =    2,
  co_varnames =     ('self',),
  depth =           2,
  co_consts = (
   None,
   codetree(
   co_argcount =     0,
   co_cellvars =     (),
   co_code =         b'\x88\x00\x00S',
   co_filename =     'super_closure.py',
   co_firstlineno =  8,
   co_flags =        19,
   co_freevars =     ('self',),
   co_kwonlyargcount =0,
   co_lnotab =       b'',
   co_name =         '<lambda>',
   co_names =        (),
   co_nlocals =      0,
   co_stacksize =    1,
   co_varnames =     (),
   depth =           3,
   co_consts = (
    )),
   )),
  )),
 A,
 None,
 ))
>>>

and disassembly:

>>>
>>> print( t.dis() )
  3           0 LOAD_BUILD_CLASS
              1 LOAD_CONST               0 (<code object A at
0x2a987af9b0, file "super_closure.py", line 3>)
              4 MAKE_FUNCTION            0
              7 LOAD_CONST               1 ('A')
             10 LOAD_NAME                0 (object)
             13 CALL_FUNCTION            3
             16 STORE_NAME               1 (A)

  9          19 LOAD_NAME                1 (A)
             22 CALL_FUNCTION            0
             25 LOAD_ATTR                2 (foo)
             28 CALL_FUNCTION            0
             31 POP_TOP
             32 LOAD_CONST               2 (None)
             35 RETURN_VALUE

      3           0 LOAD_FAST                0 (__locals__)
                  3 STORE_LOCALS
                  4 LOAD_NAME                0 (__name__)
                  7 STORE_NAME               1 (__module__)

      4          10 LOAD_CLOSURE             0 (__class__)
                 13 BUILD_TUPLE              1
                 16 LOAD_CONST               0 (<code object foo at
0x2a987afd30, file "super_closure.py", line 4>)
                 19 MAKE_CLOSURE             0
                 22 STORE_NAME               2 (foo)
                 25 LOAD_CLOSURE             0 (__class__)
                 28 RETURN_VALUE

          5           0 LOAD_GLOBAL              0 (super)
                      3 CALL_FUNCTION            0
                      6 RETURN_VALUE

          8           7 LOAD_CLOSURE             0 (self)
                     10 BUILD_TUPLE              1
                     13 LOAD_CONST               1 (<code object
<lambda> at 0x2a984c0530, file "super_closure.py", line 8>)
                     16 MAKE_CLOSURE             0
                     19 POP_TOP

              8           0 LOAD_DEREF               0 (self)
                          3 RETURN_VALUE

>>>
msg76095 - (view) Author: kai zhu (kaizhu) Date: 2008-11-20 11:10
same thing, except w/ closure commented out (& everything is happy)

################################
# super_ok.py
class A(object):
  def foo(self):
    return super()
    # comment the closure below
    # & SystemError goes away
    # lambda: self
A().foo()
################################

>>>
>>> s = open("super_ok.py").read()
>>> c = compile(s, "super_ok.py", "exec")
>>> t = py3to2.codetree(t)
>>> print( t )
codetree(
co_argcount =     0,
co_cellvars =     (),
co_code =        
b'Gd\x00\x00\x84\x00\x00d\x01\x00e\x00\x00\x83\x03\x00Z\x01\x00e\x01\x00\x83\x00\x00j\x02\x00\x83\x00\x00\x01d\x02\x00S',
co_filename =     'super_closure.py',
co_firstlineno =  3,
co_flags =        64,
co_freevars =     (),
co_kwonlyargcount =0,
co_lnotab =       b'\x13\x06',
co_name =         '<module>',
co_names =        ('object', 'A', 'foo'),
co_nlocals =      0,
co_stacksize =    4,
co_varnames =     (),
depth =           0,
co_consts = (
 codetree(
 co_argcount =     1,
 co_cellvars =     ('__class__',),
 co_code =        
b'|\x00\x00Ee\x00\x00Z\x01\x00\x87\x00\x00f\x01\x00d\x00\x00\x86\x00\x00Z\x02\x00\x87\x00\x00S',
 co_filename =     'super_closure.py',
 co_firstlineno =  3,
 co_flags =        2,
 co_freevars =     (),
 co_kwonlyargcount =0,
 co_lnotab =       b'\n\x01',
 co_name =         'A',
 co_names =        ('__name__', '__module__', 'foo'),
 co_nlocals =      1,
 co_stacksize =    2,
 co_varnames =     ('__locals__',),
 depth =           1,
 co_consts = (
  codetree(
  co_argcount =     1,
  co_cellvars =     ('self',),
  co_code =        
b't\x00\x00\x83\x00\x00S\x87\x00\x00f\x01\x00d\x01\x00\x86\x00\x00\x01',
  co_filename =     'super_closure.py',
  co_firstlineno =  4,
  co_flags =        3,
  co_freevars =     ('__class__',),
  co_kwonlyargcount =0,
  co_lnotab =       b'\x00\x01\x07\x03',
  co_name =         'foo',
  co_names =        ('super',),
  co_nlocals =      1,
  co_stacksize =    2,
  co_varnames =     ('self',),
  depth =           2,
  co_consts = (
   None,
   codetree(
   co_argcount =     0,
   co_cellvars =     (),
   co_code =         b'\x88\x00\x00S',
   co_filename =     'super_closure.py',
   co_firstlineno =  8,
   co_flags =        19,
   co_freevars =     ('self',),
   co_kwonlyargcount =0,
   co_lnotab =       b'',
   co_name =         '<lambda>',
   co_names =        (),
   co_nlocals =      0,
   co_stacksize =    1,
   co_varnames =     (),
   depth =           3,
   co_consts = (
    )),
   )),
  )),
 A,
 None,
 ))
>>>
>>> print( t.dis() )
  3           0 LOAD_BUILD_CLASS
              1 LOAD_CONST               0 (<code object A at
0x2a987af2b0, file "super_closure.py", line 3>)
              4 MAKE_FUNCTION            0
              7 LOAD_CONST               1 ('A')
             10 LOAD_NAME                0 (object)
             13 CALL_FUNCTION            3
             16 STORE_NAME               1 (A)

  9          19 LOAD_NAME                1 (A)
             22 CALL_FUNCTION            0
             25 LOAD_ATTR                2 (foo)
             28 CALL_FUNCTION            0
             31 POP_TOP
             32 LOAD_CONST               2 (None)
             35 RETURN_VALUE

      3           0 LOAD_FAST                0 (__locals__)
                  3 STORE_LOCALS
                  4 LOAD_NAME                0 (__name__)
                  7 STORE_NAME               1 (__module__)

      4          10 LOAD_CLOSURE             0 (__class__)
                 13 BUILD_TUPLE              1
                 16 LOAD_CONST               0 (<code object foo at
0x2a987af4b0, file "super_closure.py", line 4>)
                 19 MAKE_CLOSURE             0
                 22 STORE_NAME               2 (foo)
                 25 LOAD_CLOSURE             0 (__class__)
                 28 RETURN_VALUE

          5           0 LOAD_GLOBAL              0 (super)
                      3 CALL_FUNCTION            0
                      6 RETURN_VALUE

          8           7 LOAD_CLOSURE             0 (self)
                     10 BUILD_TUPLE              1
                     13 LOAD_CONST               1 (<code object
<lambda> at 0x2a987af5b0, file "super_closure.py", line 8>)
                     16 MAKE_CLOSURE             0
                     19 POP_TOP

              8           0 LOAD_DEREF               0 (self)
                          3 RETURN_VALUE

>>>
msg76096 - (view) Author: kai zhu (kaizhu) Date: 2008-11-20 11:18
oops, sorry reprinted the same code ^^;;; ignore previous post, & use
this: (sorry again for mucking up this page)

################################
# super_ok.py
class A(object):
  def foo(self):
    return super()
    # comment the closure below
    # & SystemError goes away
    # lambda: self
A().foo()
################################

>>> s = open("super_ok.py").read(); c = compile(s, "super_ok.py",
"exec"); t = py3to2.codetree(c)
>>> print( t )
codetree(
co_argcount =     0,
co_cellvars =     (),
co_code =        
b'Gd\x00\x00\x84\x00\x00d\x01\x00e\x00\x00\x83\x03\x00Z\x01\x00e\x01\x00\x83\x00\x00j\x02\x00\x83\x00\x00\x01d\x02\x00S',
co_filename =     'super_ok.py',
co_firstlineno =  3,
co_flags =        64,
co_freevars =     (),
co_kwonlyargcount =0,
co_lnotab =       b'\x13\x06',
co_name =         '<module>',
co_names =        ('object', 'A', 'foo'),
co_nlocals =      0,
co_stacksize =    4,
co_varnames =     (),
depth =           0,
co_consts = (
 codetree(
 co_argcount =     1,
 co_cellvars =     ('__class__',),
 co_code =        
b'|\x00\x00Ee\x00\x00Z\x01\x00\x87\x00\x00f\x01\x00d\x00\x00\x86\x00\x00Z\x02\x00\x87\x00\x00S',
 co_filename =     'super_ok.py',
 co_firstlineno =  3,
 co_flags =        2,
 co_freevars =     (),
 co_kwonlyargcount =0,
 co_lnotab =       b'\n\x01',
 co_name =         'A',
 co_names =        ('__name__', '__module__', 'foo'),
 co_nlocals =      1,
 co_stacksize =    2,
 co_varnames =     ('__locals__',),
 depth =           1,
 co_consts = (
  codetree(
  co_argcount =     1,
  co_cellvars =     (),
  co_code =         b't\x00\x00\x83\x00\x00S',
  co_filename =     'super_ok.py',
  co_firstlineno =  4,
  co_flags =        3,
  co_freevars =     ('__class__',),
  co_kwonlyargcount =0,
  co_lnotab =       b'\x00\x01',
  co_name =         'foo',
  co_names =        ('super',),
  co_nlocals =      1,
  co_stacksize =    1,
  co_varnames =     ('self',),
  depth =           2,
  co_consts = (
   None,
   )),
  )),
 A,
 None,
 ))
>>> print( t.dis() )
  3           0 LOAD_BUILD_CLASS
              1 LOAD_CONST               0 (<code object A at
0x2a987afd30, file "super_ok.py", line 3>)
              4 MAKE_FUNCTION            0
              7 LOAD_CONST               1 ('A')
             10 LOAD_NAME                0 (object)
             13 CALL_FUNCTION            3
             16 STORE_NAME               1 (A)

  9          19 LOAD_NAME                1 (A)
             22 CALL_FUNCTION            0
             25 LOAD_ATTR                2 (foo)
             28 CALL_FUNCTION            0
             31 POP_TOP
             32 LOAD_CONST               2 (None)
             35 RETURN_VALUE

      3           0 LOAD_FAST                0 (__locals__)
                  3 STORE_LOCALS
                  4 LOAD_NAME                0 (__name__)
                  7 STORE_NAME               1 (__module__)

      4          10 LOAD_CLOSURE             0 (__class__)
                 13 BUILD_TUPLE              1
                 16 LOAD_CONST               0 (<code object foo at
0x2a984c0530, file "super_ok.py", line 4>)
                 19 MAKE_CLOSURE             0
                 22 STORE_NAME               2 (foo)
                 25 LOAD_CLOSURE             0 (__class__)
                 28 RETURN_VALUE

          5           0 LOAD_GLOBAL              0 (super)
                      3 CALL_FUNCTION            0
                      6 RETURN_VALUE

>>>
msg76103 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-11-20 14:16
In a running frame, f->f_localplus is a vector composed of:
- the values of the local variables
- the cells containing variables used in a nested closure.
- the values of free variables defined in a outer scope.

super() needs to access the free var containing the enclosing class
object, but forgets to account for the number of cells...

The attached patch corrects the problem.
msg76104 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-11-20 14:19
Yet another release blocker for Barry. Good work, Amaury.
msg76129 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-11-20 18:57
The patch looks good to me.
msg76133 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2008-11-20 20:02
patch applied; r67299
History
Date User Action Args
2008-11-20 20:02:17barrysetstatus: open -> closed
messages: + msg76133
2008-11-20 18:58:05brett.cannonsetkeywords: - needs review
stage: patch review -> commit review
2008-11-20 18:57:51brett.cannonsetnosy: + brett.cannon
messages: + msg76129
2008-11-20 14:19:34christian.heimessetnosy: + christian.heimes, barry
messages: + msg76104
priority: release blocker
assignee: barry
components: - Build
resolution: accepted
stage: patch review
2008-11-20 14:16:22amaury.forgeotdarcsetkeywords: + needs review, patch
files: + super-withcell.patch
messages: + msg76103
nosy: + amaury.forgeotdarc
2008-11-20 11:18:50kaizhusetmessages: + msg76096
2008-11-20 11:10:28kaizhusetmessages: + msg76095
2008-11-20 10:59:15kaizhusetmessages: + msg76094
2008-11-20 10:29:19kaizhucreate