This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Zero argument super() does not function properly inside generator expressions
Type: Stage:
Components: Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: TobiasHT, carlosdamazio, josh.r, serhiy.storchaka, tritium
Priority: normal Keywords:

Created on 2021-12-25 03:25 by tritium, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
superbug.py tritium, 2021-12-25 03:25 Minimal example of the error.
superbug-fixd.py carlosdamazio, 2021-12-25 15:07
Messages (7)
msg409166 - (view) Author: Alex Walters (tritium) * Date: 2021-12-25 03:25
When zero argument super is used inside a generator expression, it raises 'TypeError: super(type, obj): obj must be an instance or subtype of type'.

I can find no information on if this is working as intended, a bug, or a known issue, and the error isn't helping.  Attached is a minimal example of the error.
msg409171 - (view) Author: Carlos Damazio (carlosdamazio) * Date: 2021-12-25 15:07
By explicitly sending a type and object to the super function, it should work as usual, like in the file that I've just sent. I've seen this behavior before, it's related to Python's module reload function, which returns a new reference to the reloaded module. This issue is not widely known though, should be documented.
msg409178 - (view) Author: TobiasHT (TobiasHT) * Date: 2021-12-26 05:28
I'm not so sure, but I think this characteristic was carried from python 2. I guess it's not well documented because they assumed that people would carry on the knowledge to python 3. To be on the safe side, I always pass the arguments to super.

https://docs.python.org/2/library/functions.html#super
msg409199 - (view) Author: Carlos Damazio (carlosdamazio) * Date: 2021-12-26 13:13
Normally, users assume it's safe to use `super` without explicit arguments, until an undefined behavior happens, such as now. The only thing that glances into this issue is the observation in the docs you've provided that omitting the second argument (self), `super` returns an unbounded object, which is a super object.

I mean, there are 2 alternatives: this issue is related to a lower level implementation and it's another way to solve it (of which needs investigation of course) or state that it's required to provide such arguments in the docs.

https://docs.python.org/3.9/library/functions.html#super

In the newer docs, we are assuming that `super()` is the same as `super(cls, self)`, but clearly it's not.
msg409249 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2021-12-28 05:11
Carlos: This has nothing to do with reloading (as Alex's repro shows, no reload calls are made).

super() *should* behave the same as super(CLASS_DEFINED_IN, self), and it looks like the outer function is doing half of what it must do to make no-arg super() work in the genexpr (dis.dis reports that __class__ is being loaded, and a closure constructed from the genexpr that includes it, so __class__, which no-arg super pulls from closure scope to get its first argument, is there).

The problem is that super() *also* assumes the first argument to the function is self, and a genexpr definitionally receives just one argument, the iterator (the outermost one for genexprs with nested loops). So no-arg super is doing the equivalent of:

super(__class__, iter(vars))

when it should be doing:

super(__class__, self)

Only way to fix it I can think of would be one of:

1. Allow a genexpr to receive multiple arguments to support this use case (ugly, requires significant changes to current design of genexprs and probably super() too)
2. Somehow teach super() to pull self (positional argument #1 really; super() doesn't care about names) from closure scope (and make the compiler put self in the closure scope when it builds the closure) when run in a genexpr.

Both options seem... sub-optimal. Better suggestions welcome. Note that the same problem affects the various forms of comprehension as well (this isn't specific to the lazy design of genexprs; listcomps have the same problem).
msg409250 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-12-28 06:48
I concur with Josh. super() uses a lot of magic, and in case of comprehensions it fails in interesting way.

Most common cases in which super() does not work:

1. Outside of a function defined in a class.
2. In a static method.
3. In inner function.
4. In comprehensions and generator expressions.

Idea #2 could help in cases 3 and 4.
msg409263 - (view) Author: Carlos Damazio (carlosdamazio) * Date: 2021-12-28 18:28
Josh: My mistake, I've seen a similar issue, then. And agreed, I think #2 is a great candidate since we don't need to re-design existing structures. I don't know a better option... yet.
History
Date User Action Args
2022-04-11 14:59:53adminsetgithub: 90333
2021-12-28 18:28:25carlosdamaziosetmessages: + msg409263
2021-12-28 06:48:57serhiy.storchakasetmessages: + msg409250
2021-12-28 05:11:49josh.rsetnosy: + josh.r
messages: + msg409249
2021-12-26 13:13:01carlosdamaziosetmessages: + msg409199
2021-12-26 05:28:37TobiasHTsetnosy: + TobiasHT
messages: + msg409178
2021-12-25 15:07:37carlosdamaziosetfiles: + superbug-fixd.py
nosy: + carlosdamazio
messages: + msg409171

2021-12-25 09:37:23serhiy.storchakasetnosy: + serhiy.storchaka
2021-12-25 03:25:41tritiumcreate