This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Add class binding to unbound super objects for allowing autosuper with class methods
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.11
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: gvanrossum, maggyero, rhettinger
Priority: normal Keywords: patch

Created on 2021-05-09 17:02 by maggyero, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 26009 closed maggyero, 2021-05-09 17:05
Messages (16)
msg393326 - (view) Author: Géry (maggyero) * Date: 2021-05-09 17:02
A use case of one-argument `super` (aka unbound `super`) is Guido van Rossum’s autosuper described in his 2002 article [*Unifying types and classes in Python 2.2*](https://www.python.org/download/releases/2.2.3/descrintro/#cooperation).

It works with functions, but not with `classmethod` as Michele Simionato noted in his 2008 article [*Things to Know About Python Super*](https://www.artima.com/weblogs/viewpost.jsp?thread=236278).

I suggest fixing this by updating the method `super.__get__` to bind an unbound `super` object to the argument `owner` when there is no argument `instance` to bind to. Here is the patch applied to the C function [super_descr_get](https://github.com/python/cpython/blob/v3.9.5/Objects/typeobject.c#L8029-L8061) in Objects/typeobject.c, given in pure Python for better readability:

```python
    def __get__(self, instance, owner=None):
        if instance is None and owner is None:
            raise TypeError('__get__(None, None) is invalid')
-       if instance is None or self.__self__ is not None:
+       if self.__self__ is not None:
            return self
+       if instance is None:
+           return type(self)(self.__thisclass__, owner)
        return type(self)(self.__thisclass__, instance)
```

Demonstration:

```python
>>> class A:
...     def f(self): return 'A.f'
...     @classmethod
...     def g(cls): return 'A.g'
... 
>>> class B(A):
...     def f(self): return 'B.f ' + self.__super.f()
...     @classmethod
...     def g(cls): return 'B.g ' + cls.__super.g()
... 
>>> B._B__super = super(B)  # the CURRENT broken version of super
>>> print(B().f())  # function succeeds (instance binding)
B.f A.f
>>> print(B.g())    # classmethod fails (no binding)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in g
AttributeError: 'super' object has no attribute 'g'
>>> B._B__super = super(B)  # the PROPOSED fixed version of super
>>> print(B().f())  # function succeeds (instance binding)
B.f A.f
>>> print(B.g())    # classmethod succeeds (class binding)
B.g A.g
```
msg393349 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-05-10 03:56
Do we have any meaningful examples to show that this is desired and useful? 

The primary use case for classmethods is to serve as alternate constructors that return new instances.  That doesn't really lend itself to extending in a subclass.  User's can already call a parent class directly without super(), but I don't think I've ever encountered a single example of someone doing so with a classmethod such as dict.fromkeys() or datetime.fromtimestamp().
msg415886 - (view) Author: Géry (maggyero) * Date: 2022-03-23 17:22
Apologies for the long delay.

> Do we have any meaningful examples to show that this is desired and useful?

A use case of `super()` in a `classmethod` that comes to mind is for parameterized factory methods. It is a variation of the classic factory method design pattern that lets the factory method of a creator creates *multiple* products according to a parameter identifying the product to create. Overriding the factory method lets you change or extend the products that are created, by mapping existing identifiers to different products or introducing new identifiers for new products (cf. GoF’s book *Design Patterns*, section ‘Factory Method’, subsection ‘Implementation’, paragraph 2):

```
>>> class MyCreator:
...     @classmethod
...     def make(cls, product_id):
...         if product_id == 'mine': return MyProduct(creator=cls)
...         if product_id == 'yours': return YourProduct(creator=cls)
...         if product_id == 'theirs': return TheirProduct(creator=cls)
...         raise ValueError('product_id {!r} not supported'.format(product_id))
... 
>>> class YourCreator(MyCreator):
...     @classmethod
...     def make(cls, product_id):
...         if product_id == 'mine': return YourProduct(creator=cls)
...         if product_id == 'yours': return MyProduct(creator=cls)
...         return super(YourCreator, cls).make(product_id)
... 
>>> class BaseProduct:
...     def __init__(self, creator): self._creator = creator
...     def __repr__(self):
...         return '{}(creator={})'.format(
...             type(self).__name__, self._creator.__name__)
... 
>>> class MyProduct(BaseProduct): pass
... 
>>> class YourProduct(BaseProduct): pass
... 
>>> class TheirProduct(BaseProduct): pass
... 
>>> MyCreator.make('mine')
MyProduct(creator=MyCreator)
>>> YourCreator.make('mine')
YourProduct(creator=YourCreator)
```
msg416034 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2022-03-26 01:42
Guido, what do you think about this proposal?  

Personally, I'm dubious about changing the meaning of the arguments between code paths.  The callee has no way to distinguish which meaning was intended.  And adding classmethod() support in super_descr_get() would create tight coupling where there should be separate concerns (no other descriptor call is classmethod specific).  

On StackOverflow, there has been some mild interest in the interactions between super() and classmethod():

* https://stackoverflow.com/questions/64637174
* https://stackoverflow.com/questions/1269217
* https://stackoverflow.com/questions/1817183

The OP's proposed use case is mildly plausible though I've never seen it the arise in practice.  The GoF factory pattern is typically implemented in a function rather than in the class itself — for example the open() function returns one of two different classes depending on the text or binary file mode.  It is rare to see __new__ used to fork off different types based on a parameter.  Presumably, it would be even more rare with a classmethod.  Also, it's not clear that the GoF pattern would mesh well with our cooperative super() since the upstream parent class isn't known in advance.
msg416035 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2022-03-26 04:45
I’m sorry, my brain hurts when trying to understand my own code for super.
Hopefully someone younger can look at this.--
--Guido (mobile)
msg416043 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2022-03-26 06:44
Will leave this open for a few days to see if anyone else steps forward to make a case for or against this proposal.
msg416137 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2022-03-27 22:09
So IIUC the "autosuper" idea is to assign a special instance of super to self.__super, so that you can write self.__super.method(...) to invoke a super method, using the magic of __private variables, instead of having to write super(classname, self).method(...).

That was perhaps a good idea 20 years ago, but nowadays you can use argument-less super(), so you can write super().method(...), and AFAICT that works for class methods too.

I don't think we need two ways to do it.
msg416271 - (view) Author: Géry (maggyero) * Date: 2022-03-29 16:10
Thanks for the review.

@rhettinger

> And adding classmethod() support in super_descr_get() would create tight coupling where there should be separate concerns (no other descriptor call is classmethod specific).

The descriptors `super`, `property`, and functions are already `classmethod` specific since their `__get__(instance, owner=None)` methods return `self` if `instance` is `None`, aren’t they?

> The OP's proposed use case is mildly plausible though I've never seen it the arise in practice.

I agree that the parameterized factory method use case might be too rare to be compelling.

@gvanrossum

> That was perhaps a good idea 20 years ago, but nowadays you can use argument-less super()

Yes this proposal is likely too late. I found your autosuper solution quite elegant (no compiler magic) so I wanted to make it work with `classmethod` too after I realized it doesn’t, thanks to Michele’s article.
msg416272 - (view) Author: Géry (maggyero) * Date: 2022-03-29 16:21
> On StackOverflow, there has been some mild interest in the interactions between super() and classmethod():
> 
> * https://stackoverflow.com/questions/64637174
> * https://stackoverflow.com/questions/1269217
> * https://stackoverflow.com/questions/1817183

Another one: https://stackoverflow.com/q/15291302/2326961
msg416275 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2022-03-29 17:02
Thanks, let's close the issue as "won't fix".
msg416326 - (view) Author: Géry (maggyero) * Date: 2022-03-30 07:03
By the way:

> I don't think we need two ways to do it.

So do you think we could drop the support for single-argument super?

Michele said in his article:

> There is a single use case for the single argument syntax of super that I am aware of, but I think it gives more troubles than advantages. The use case is the implementation of autosuper made by Guido on his essay about new-style classes.

> If it was me, I would just remove the single argument syntax of super, making it illegal. But this would probably break someone code, so I don't think it will ever happen in Python 2.X. I did ask on the Python 3000 mailing list about removing unbound super object (the title of the thread was let's get rid of unbound super) and this was Guido's reply:

>> Thanks for proposing this -- I've been scratching my head wondering what the use of unbound super() would be. :-) I'm fine with killing it -- perhaps someone can do a bit of research to try and find out if there are any real-life uses (apart from various auto-super clones)? --- Guido van Rossum

> Unfortunaly as of now unbound super objects are still around in Python 3.0, but you should consider them morally deprecated.
msg416391 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2022-03-30 18:11
Yeah, I see no description of what you can do with an unbound super object in the docs (https://docs.python.org/3/library/functions.html#super), and experimentation with it does not reveal any useful functionality.

You may want to open a new issue for this, and we'll probably have to propose a 2-release deprecation period and start issuing a deprecation warning, in case there are nevertheless users (like autosuper clones).
msg416402 - (view) Author: Géry (maggyero) * Date: 2022-03-30 21:52
Alright, thanks. Raymond, any objections before I propose the removal of one-argument super?
msg416749 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2022-04-05 04:33
> any objections before I propose the removal of one-argument super?

AFAICT there is nothing to be gained by deprecating the one argument form.  Because it has been stable API for two decades, removing it is guaranteed to cause some disruption. So why bother?

Anthony Sottile provided this search to showing that at least a few popular projects are using the one argument form of super():

https://sourcegraph.com/search?q=context:global+%5Cbsuper%5C%28%5B%5E%2C%5C%28%5C%29%5Cn%5D%2B%5C%29+file:.py%24&patternType=regexp&case=yes
msg416773 - (view) Author: Géry (maggyero) * Date: 2022-04-05 11:59
> Anthony Sottile provided this search to showing that at least a few popular projects are using the one argument form of super():

Thanks for the link. But it gives lots of false positives. Only two popular projects (> 1k stars) use autosuper (urwid and evennia):

https://sourcegraph.com/search?q=context:global+setattr%5C%28.*super%5C%28.*%5C%29%5C%29+file:%5C.py%24+-file:test.*%5C.py%24&patternType=regexp&case=yes

The remaining projects use one-argument super incorrectly, as `super(cls).method()`, which looks up the method directly on class `super`:

https://sourcegraph.com/search?q=context:global+content:%5B%5E_%5Dsuper%5C%28%5Cw%2B%5C%29%5B%5E:%5D+file:%5C.py%24+-file:test.*%5C.py%24&patternType=regexp&case=yes

It is either a loud bug, which raises an `AttributeError`:

```
>>> class A:
...     def f(self): pass
... 
>>> class B(A):
...     def f(self): super(B).f()
... 
>>> B().f()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in f
AttributeError: 'super' object has no attribute 'f'
```

Or worse with `super(cls).__init__()` (99% of the cases), it is a SILENT bug, which call the constructor of class `super` instead of the parent constructor, leaving the object in an incompletely initialized state:

```
>>> class A:
...     def __init__(self): print('hello')
... 
>>> class B(A):
...     def __init__(self): super(B).__init__()
... 
>>> A()
hello
<__main__.A object at 0x10926e460>
>>> B()
<__main__.B object at 0x10926e520>
```
msg416823 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2022-04-05 22:15
At this point I think it's worth filing a new bug proposing to deprecate 1-arg super(), pointing out the broken usages that search found.
History
Date User Action Args
2022-04-11 14:59:45adminsetgithub: 88256
2022-04-05 22:15:37gvanrossumsetmessages: + msg416823
2022-04-05 11:59:10maggyerosetmessages: + msg416773
2022-04-05 04:33:27rhettingersetmessages: + msg416749
2022-03-30 21:52:36maggyerosetmessages: + msg416402
2022-03-30 18:11:51gvanrossumsetmessages: + msg416391
2022-03-30 07:03:56maggyerosetmessages: + msg416326
2022-03-29 17:02:56gvanrossumsetstatus: open -> closed
resolution: wont fix
messages: + msg416275

stage: patch review -> resolved
2022-03-29 16:21:28maggyerosetmessages: + msg416272
2022-03-29 16:10:33maggyerosetmessages: + msg416271
2022-03-27 22:09:27gvanrossumsetmessages: + msg416137
2022-03-26 06:44:33rhettingersetassignee: rhettinger
messages: + msg416043
versions: - Python 3.9, Python 3.10
2022-03-26 04:45:21gvanrossumsetmessages: + msg416035
2022-03-26 01:42:20rhettingersetnosy: + gvanrossum
messages: + msg416034
2022-03-23 17:22:45maggyerosetmessages: + msg415886
2021-05-10 03:56:56rhettingersetmessages: + msg393349
2021-05-10 03:09:55rhettingersetnosy: + rhettinger
2021-05-09 17:05:51maggyerosetkeywords: + patch
stage: patch review
pull_requests: + pull_request24660
2021-05-09 17:02:11maggyerocreate