--- pep-3135-n.txt 2009-03-12 04:17:21.988000000 +1100 +++ pep-3135.txt 2009-03-12 04:28:34.451000000 +1100 @@ -3,13 +3,14 @@ Version: $Revision$ Last-Modified: $Date$ Author: Calvin Spealman , - Tim Delaney -Status: Draft + Tim Delaney , + Lie Ryan +Status: Accepted Type: Standards Track Content-Type: text/x-rst Created: 28-Apr-2007 Python-Version: 3.0 -Post-History: 28-Apr-2007, 29-Apr-2007 (1), 29-Apr-2007 (2), 14-May-2007 +Post-History: 28-Apr-2007, 29-Apr-2007 (1), 29-Apr-2007 (2), 14-May-2007, 12-Mar-2009 Numbering Note ============== @@ -17,11 +18,6 @@ This PEP started its life as PEP 367. Since it is now targeted for Python 3000, it has been moved into the 3xxx space. -NOTE: This PEP needs to be rewritten to match reality. The actual -implementation is quite different than what is described here; -effectively, super() equals to super(C, self) where C is the current -class and self is the first argument of the current function. - Abstract ======== @@ -32,19 +28,12 @@ The premise of the new super usage suggested is as follows:: - super.foo(1, 2) + super().foo(1, 2) to replace the old:: super(Foo, self).foo(1, 2) -and the current ``__builtin__.super`` be aliased to ``__builtin__.__super__`` -(with ``__builtin__.super`` to be removed in Python 3.0). - -It is further proposed that assignment to ``super`` become a ``SyntaxError``, -similar to the behaviour of ``None``. - - Rationale ========= @@ -58,80 +47,48 @@ ============= Within the specification section, some special terminology will be used to -distinguish similar and closely related concepts. "super type" will refer to -the actual builtin type named "super". A "super instance" is simply an instance -of the super type, which is associated with a class and possibly with an -instance of that class. +distinguish similar and closely related concepts. "super class" will refer to +the actual builtin class named "super". A "super instance" is simply an +instance of the super class, which is associated with another class and +possibly with an instance of that class. Because the new ``super`` semantics are not backwards compatible with Python 2.5, the new semantics will require a ``__future__`` import:: from __future__ import new_super -The current ``__builtin__.super`` will be aliased to ``__builtin__.__super__``. -This will occur regardless of whether the new ``super`` semantics are active. -It is not possible to simply rename ``__builtin__.super``, as that would affect -modules that do not use the new ``super`` semantics. In Python 3.0 it is -proposed that the name ``__builtin__.super`` will be removed. - Replacing the old usage of super, calls to the next class in the MRO (method -resolution order) can be made without explicitly creating a ``super`` -instance (although doing so will still be supported via ``__super__``). Every -function will have an implicit local named ``super``. This name behaves -identically to a normal local, including use by inner functions via a cell, -with the following exceptions: +resolution order) can be made without explicitly passing the class object +(although doing so will still be supported via ``__super__``). Every function +will have a cell named ``__class__`` that contains the class object that the +function is defined in. -1. Assigning to the name ``super`` will raise a ``SyntaxError`` at compile time; +The new syntax:: -2. Calling a static method or normal function that accesses the name ``super`` - will raise a ``TypeError`` at runtime. + super() -Every function that uses the name ``super``, or has an inner function that -uses the name ``super``, will include a preamble that performs the equivalent -of:: +is equivalent to:: - super = __builtin__.__super__(, ) + super(__class__, ) -where ```` is the class that the method was defined in, and -```` is the first parameter of the method (normally ``self`` for +where ``__class__`` is the class that the method was defined in, and +```` is the first parameter of the method (normally ``self`` for instance methods, and ``cls`` for class methods). For static methods and normal -functions, ```` will be ``None``, resulting in a ``TypeError`` being -raised during the preamble. - -Note: The relationship between ``super`` and ``__super__`` is similar to that -between ``import`` and ``__import__``. - -Much of this was discussed in the thread of the python-dev list, "Fixing super -anyone?" [1]_. - - -Open Issues ------------ +functions, ``__class__`` is not defined, and will result in runtime ``SystemError``. +Closed Issues +------------- Determining the class object to use ''''''''''''''''''''''''''''''''''' -The exact mechanism for associating the method with the defining class is not -specified in this PEP, and should be chosen for maximum performance. For -CPython, it is suggested that the class instance be held in a C-level variable -on the function object which is bound to one of ``NULL`` (not part of a class), -``Py_None`` (static method) or a class object (instance or class method). +The class object would be taken from a cell named ``__class__``. Should ``super`` actually become a keyword? ''''''''''''''''''''''''''''''''''''''''''' -With this proposal, ``super`` would become a keyword to the same extent that -``None`` is a keyword. It is possible that further restricting the ``super`` -name may simplify implementation, however some are against the actual keyword- -ization of super. The simplest solution is often the correct solution and the -simplest solution may well not be adding additional keywords to the language -when they are not needed. Still, it may solve other open issues. - - -Closed Issues -------------- +No. It is not necessary for super to become a keyword. super used with __call__ attributes ''''''''''''''''''''''''''''''''''' @@ -158,313 +115,6 @@ In any case, with the renaming of ``__builtin__.super`` to ``__builtin__.__super__`` this issue goes away entirely. - -Reference Implementation -======================== - -It is impossible to implement the above specification entirely in Python. This -reference implementation has the following differences to the specification: - -1. New ``super`` semantics are implemented using bytecode hacking. - -2. Assignment to ``super`` is not a ``SyntaxError``. Also see point #4. - -3. Classes must either use the metaclass ``autosuper_meta`` or inherit from - the base class ``autosuper`` to acquire the new ``super`` semantics. - -4. ``super`` is not an implicit local variable. In particular, for inner - functions to be able to use the super instance, there must be an assignment - of the form ``super = super`` in the method. - -The reference implementation assumes that it is being run on Python 2.5+. - -:: - - #!/usr/bin/env python - # - # autosuper.py - - from array import array - import dis - import new - import types - import __builtin__ - __builtin__.__super__ = __builtin__.super - del __builtin__.super - - # We need these for modifying bytecode - from opcode import opmap, HAVE_ARGUMENT, EXTENDED_ARG - - LOAD_GLOBAL = opmap['LOAD_GLOBAL'] - LOAD_NAME = opmap['LOAD_NAME'] - LOAD_CONST = opmap['LOAD_CONST'] - LOAD_FAST = opmap['LOAD_FAST'] - LOAD_ATTR = opmap['LOAD_ATTR'] - STORE_FAST = opmap['STORE_FAST'] - LOAD_DEREF = opmap['LOAD_DEREF'] - STORE_DEREF = opmap['STORE_DEREF'] - CALL_FUNCTION = opmap['CALL_FUNCTION'] - STORE_GLOBAL = opmap['STORE_GLOBAL'] - DUP_TOP = opmap['DUP_TOP'] - POP_TOP = opmap['POP_TOP'] - NOP = opmap['NOP'] - JUMP_FORWARD = opmap['JUMP_FORWARD'] - ABSOLUTE_TARGET = dis.hasjabs - - def _oparg(code, opcode_pos): - return code[opcode_pos+1] + (code[opcode_pos+2] << 8) - - def _bind_autosuper(func, cls): - co = func.func_code - name = func.func_name - newcode = array('B', co.co_code) - codelen = len(newcode) - newconsts = list(co.co_consts) - newvarnames = list(co.co_varnames) - - # Check if the global 'super' keyword is already present - try: - sn_pos = list(co.co_names).index('super') - except ValueError: - sn_pos = None - - # Check if the varname 'super' keyword is already present - try: - sv_pos = newvarnames.index('super') - except ValueError: - sv_pos = None - - # Check if the callvar 'super' keyword is already present - try: - sc_pos = list(co.co_cellvars).index('super') - except ValueError: - sc_pos = None - - # If 'super' isn't used anywhere in the function, we don't have anything to do - if sn_pos is None and sv_pos is None and sc_pos is None: - return func - - c_pos = None - s_pos = None - n_pos = None - - # Check if the 'cls_name' and 'super' objects are already in the constants - for pos, o in enumerate(newconsts): - if o is cls: - c_pos = pos - - if o is __super__: - s_pos = pos - - if o == name: - n_pos = pos - - # Add in any missing objects to constants and varnames - if c_pos is None: - c_pos = len(newconsts) - newconsts.append(cls) - - if n_pos is None: - n_pos = len(newconsts) - newconsts.append(name) - - if s_pos is None: - s_pos = len(newconsts) - newconsts.append(__super__) - - if sv_pos is None: - sv_pos = len(newvarnames) - newvarnames.append('super') - - # This goes at the start of the function. It is: - # - # super = __super__(cls, self) - # - # If 'super' is a cell variable, we store to both the - # local and cell variables (i.e. STORE_FAST and STORE_DEREF). - # - preamble = [ - LOAD_CONST, s_pos & 0xFF, s_pos >> 8, - LOAD_CONST, c_pos & 0xFF, c_pos >> 8, - LOAD_FAST, 0, 0, - CALL_FUNCTION, 2, 0, - ] - - if sc_pos is None: - # 'super' is not a cell variable - we can just use the local variable - preamble += [ - STORE_FAST, sv_pos & 0xFF, sv_pos >> 8, - ] - else: - # If 'super' is a cell variable, we need to handle LOAD_DEREF. - preamble += [ - DUP_TOP, - STORE_FAST, sv_pos & 0xFF, sv_pos >> 8, - STORE_DEREF, sc_pos & 0xFF, sc_pos >> 8, - ] - - preamble = array('B', preamble) - - # Bytecode for loading the local 'super' variable. - load_super = array('B', [ - LOAD_FAST, sv_pos & 0xFF, sv_pos >> 8, - ]) - - preamble_len = len(preamble) - need_preamble = False - i = 0 - - while i < codelen: - opcode = newcode[i] - need_load = False - remove_store = False - - if opcode == EXTENDED_ARG: - raise TypeError("Cannot use 'super' in function with EXTENDED_ARG opcode") - - # If the opcode is an absolute target it needs to be adjusted - # to take into account the preamble. - elif opcode in ABSOLUTE_TARGET: - oparg = _oparg(newcode, i) + preamble_len - newcode[i+1] = oparg & 0xFF - newcode[i+2] = oparg >> 8 - - # If LOAD_GLOBAL(super) or LOAD_NAME(super) then we want to change it into - # LOAD_FAST(super) - elif (opcode == LOAD_GLOBAL or opcode == LOAD_NAME) and _oparg(newcode, i) == sn_pos: - need_preamble = need_load = True - - # If LOAD_FAST(super) then we just need to add the preamble - elif opcode == LOAD_FAST and _oparg(newcode, i) == sv_pos: - need_preamble = need_load = True - - # If LOAD_DEREF(super) then we change it into LOAD_FAST(super) because - # it's slightly faster. - elif opcode == LOAD_DEREF and _oparg(newcode, i) == sc_pos: - need_preamble = need_load = True - - if need_load: - newcode[i:i+3] = load_super - - i += 1 - - if opcode >= HAVE_ARGUMENT: - i += 2 - - # No changes needed - get out. - if not need_preamble: - return func - - # Our preamble will have 3 things on the stack - co_stacksize = max(3, co.co_stacksize) - - # Conceptually, our preamble is on the `def` line. - co_lnotab = array('B', co.co_lnotab) - - if co_lnotab: - co_lnotab[0] += preamble_len - - co_lnotab = co_lnotab.tostring() - - # Our code consists of the preamble and the modified code. - codestr = (preamble + newcode).tostring() - - codeobj = new.code(co.co_argcount, len(newvarnames), co_stacksize, - co.co_flags, codestr, tuple(newconsts), co.co_names, - tuple(newvarnames), co.co_filename, co.co_name, - co.co_firstlineno, co_lnotab, co.co_freevars, - co.co_cellvars) - - func.func_code = codeobj - func.func_class = cls - return func - - class autosuper_meta(type): - def __init__(cls, name, bases, clsdict): - UnboundMethodType = types.UnboundMethodType - - for v in vars(cls): - o = getattr(cls, v) - if isinstance(o, UnboundMethodType): - _bind_autosuper(o.im_func, cls) - - class autosuper(object): - __metaclass__ = autosuper_meta - - if __name__ == '__main__': - class A(autosuper): - def f(self): - return 'A' - - class B(A): - def f(self): - return 'B' + super.f() - - class C(A): - def f(self): - def inner(): - return 'C' + super.f() - - # Needed to put 'super' into a cell - super = super - return inner() - - class D(B, C): - def f(self, arg=None): - var = None - return 'D' + super.f() - - assert D().f() == 'DBCA' - -Disassembly of B.f and C.f reveals the different preambles used when ``super`` -is simply a local variable compared to when it is used by an inner function. - -:: - - >>> dis.dis(B.f) - - 214 0 LOAD_CONST 4 () - 3 LOAD_CONST 2 () - 6 LOAD_FAST 0 (self) - 9 CALL_FUNCTION 2 - 12 STORE_FAST 1 (super) - - 215 15 LOAD_CONST 1 ('B') - 18 LOAD_FAST 1 (super) - 21 LOAD_ATTR 1 (f) - 24 CALL_FUNCTION 0 - 27 BINARY_ADD - 28 RETURN_VALUE - -:: - - >>> dis.dis(C.f) - - 218 0 LOAD_CONST 4 () - 3 LOAD_CONST 2 () - 6 LOAD_FAST 0 (self) - 9 CALL_FUNCTION 2 - 12 DUP_TOP - 13 STORE_FAST 1 (super) - 16 STORE_DEREF 0 (super) - - 219 19 LOAD_CLOSURE 0 (super) - 22 LOAD_CONST 1 () - 25 MAKE_CLOSURE 0 - 28 STORE_FAST 2 (inner) - - 223 31 LOAD_FAST 1 (super) - 34 STORE_DEREF 0 (super) - - 224 37 LOAD_FAST 2 (inner) - 40 CALL_FUNCTION 0 - 43 RETURN_VALUE - -Note that in the final implementation, the preamble would not be part of the -bytecode of the method, but would occur immediately following unpacking of -parameters. - - Alternative Proposals ===================== @@ -491,16 +141,6 @@ This proposal relies on sys._getframe(), which is not appropriate for anything except a prototype implementation. - -super(__this_class__, self) ---------------------------- - -This is nearly an anti-proposal, as it basically relies on the acceptance of -the __this_class__ PEP, which proposes a special name that would always be -bound to the class within which it is used. If that is accepted, __this_class__ -could simply be used instead of the class' name explicitly, solving the name -binding issues [2]_. - self.__super__.foo(\*args) -------------------------- @@ -561,6 +201,8 @@ History ======= +12-Mar-2009 - Updated to reflect the current state of implementation. + 29-Apr-2007 - Changed title from "Super As A Keyword" to "New Super" - Updated much of the language and added a terminology section for clarification in confusing places.