The documentation for super() at https://docs.python.org/3.8/library/functions.html#super does not actually say what super() does. It only says "Return a proxy object that delegates method calls to a parent or sibling class of type" and then gives a bunch of use cases and examples.

If there is one place where we should define exactly what super() does (as opposed to give guidance on how to use it), the stdlib reference should be it.

The docs do say what super does: it returns "a proxy object that delegates method calls to a parent or sibling class of type", just as you quoted. That concise description is (almost) completely accurate and precise.

(I say *almost* because it's not just method calls that it works with, but any attribute lookup.)

What more do you want? That's not a rhetorical question.

I'm not saying that the docs are perfect or cannot be improved, but they look pretty good to me: they tell you what super does, they tell you why you might use it, and show how to use it. What's missing? Again, not a rhetorical question.

I understand that super is a very advanced corner of the language: there's a lot of necessary technical jargon in the docs, e.g.:

- One already needs to have a good understanding of what super *does* in order to make sense of the sentence describing what it *is*, so there's an element of circular reasoning needed.

- The reader needs to understand that super isn't magical, it just returns an object like any other function, and understand what "proxy object" means, as well as delegation.

- And have an understanding of how inheritance and the MRO work in Python, and the difference between bound and unbound methods/proxies.

I think that to the experienced reader who knows these concepts, the docs should be pretty clear and complete.

I'm having a hard time seeing what you believe is missing from the docs. Can you explain further?

Perhaps there ought to be a "gentle guide to super" somewhere, and the docs could link to that?

> It only says "Return a proxy object that delegates method 
> calls to a parent or sibling class of type" and then gives
> a bunch of use cases and examples."  

That wording seems very reasonable to me.

> What more do you want?

Mainly: it says "a parent or sibling class of *type*" but it doesn't explain which class it actually uses.

And the sentence "The __mro__ attribute of the type lists the method resolution search order used by both getattr() and super()" is even wrong or at least confusing: what matters is not the MRO of the type (the first argument to super()) but the MRO of the object (the second argument to super()).

The zero-argument form super() is not explained at all.

> Perhaps there ought to be a "gentle guide to super" somewhere, and the docs could link to that?

There are plenty of guides like that and in fact that docs already link to https://rhettinger.wordpress.com/2011/05/26/super-considered-super/

On Fri, Jun 07, 2019 at 08:00:34AM +0000, Jeroen Demeyer wrote:
> 
> Jeroen Demeyer <J.Demeyer@UGent.be> added the comment:
> 
> > What more do you want?
> 
> Mainly: it says "a parent or sibling class of *type*" but it doesn't 
> explain which class it actually uses.

Only one of the two arguments is called "type". The other is called 
"object-or-type".

> And the sentence "The __mro__ attribute of the type lists the method 
> resolution search order used by both getattr() and super()" is even 
> wrong or at least confusing: what matters is not the MRO of the type 
> (the first argument to super()) but the MRO of the object (the second 
> argument to super()).

The sentence doesn't talk about the MRO of *type* (the first argument), 
it talks about the __mro__ attribute. It is correct. The __mro__ 
attribute is on the type, not the instance:

py> int.__mro__
(<class 'int'>, <class 'object'>)

py> int().__mro__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute '__mro__'

super() look-ups never start at the instance, they delegate to the 
superclass (or superclasses), not back to the current class, or the 
instance itself.

> The zero-argument form super() is not explained at all.

Yes it is. Look at the example given:

    super().method(arg)    # This does the same thing as:
                           # super(C, self).method(arg)

> Only one of the two arguments is called "type". The other is called 
"object-or-type".

I'm having problems with the first word of "a parent or sibling class of type". The most important part of super() is to *which* class that attribute lookups are delegated to and this is not explained.

> The __mro__ attribute is on the type, not the instance:

Sorry for that. I meant to say:

And the sentence "The __mro__ attribute of the type lists the method resolution search order used by both getattr() and super()" is even wrong or at least confusing: what matters is not the MRO of the type (the first argument to super()) but the MRO of ***the type of*** the object (the second argument to super()).

> Yes it is. Look at the example given:

An example is not an explanation. But it's true, this is the least of my problems with this doc.

> The sentence doesn't talk about the MRO of *type* (the first argument), 
it talks about the __mro__ attribute.

If you have to explain in a bpo issue how the doc should be read, that proves exactly my point that it's confusing. The fact that it's technically correct if you read it the right way is irrelevant.

On Fri, Jun 07, 2019 at 09:57:11AM +0000, Jeroen Demeyer wrote:

> I'm having problems with the first word of "a parent or sibling class of type". 

The first word is "a". Did you mean something else or did you mean it 
literally?

If the second case, we have to talk about *a* parent class rather than 
*the* parent class, since Python supports multiple inheritance and a 
class can have more than one parent class.

> The most important part of super() is to *which* class that attribute 
> lookups are delegated to and this is not explained.

Lookups are delegated to a parent or sibling class, exactly as the 
documentation says. Which class that is (whether a parent or sibling) 
can only be determined at runtime.

> And the sentence "The __mro__ attribute of the type lists the method 
> resolution search order used by both getattr() and super()" is even 
> wrong or at least confusing: what matters is not the MRO of the type 
> (the first argument to super()) but the MRO of ***the type of*** the 
> object (the second argument to super()).

I believe that is still wrong.

What matters is the __mro__ attribute of the first argument. It matters 
because that is how the MRO actually is searched.

The MRO of the second argument is not relevant, except to the degree 
that it overlaps with the __mro__ attribute of the first. And it will 
always overlaps, because the second argument must be an instance or 
subclass of the first. But not necessarily *directly* of the first:

object > Parent > A > B > C > D > E > F > instance = F()

If the method is defined in class C, then the super call in C should be 
written:

    class C(B):
        def method(self, arg):
            super().method(arg)  # like super(C, self).method(arg)

and the lookup will start at C.__mro__[1], namely B, precisely as 
expected. If it started back at the type of the instance (self), namely 
class F, the C.method would recursively call itself over and over and 
over again.

If you don't believe me, you can simulate super() working the way you 
suggest by doing this:

# -----%<-----

class A(object):
    def method(self):
        print("from class A")

class B(A):
    def method(self):
        print("bad method")
        # This is bad, don't do it!
        super(type(self), self).method()

class C(B): pass

C().method()

# -----%<-----

For multiple inheritance:

    class C(B, X, Y, Z): ...

we can't generally tell in advance which will be looked at next, since 
that depends on the precise details of the inheritance diagram. But 
whatever the class is, at this point, it is still the __mro__ of C that 
is used, not the MRO of the instance.

The bottom line: what matters is <first argument>.__mro__, exactly as 
documented.

> If you have to explain in a bpo issue how the doc should be read, that 
> proves exactly my point that it's confusing. The fact that it's 
> technically correct if you read it the right way is irrelevant.

Do you expect the docs to be technically correct when read wrongly? How 
else should people read the docs, except the right way?

If you have *concrete* suggestions for improvements, of course we will 
consider them. I'll start with two concrete improvements:

1. Explicitly document that the zero-argument version is equivalent
to super(__class__, <first argument (usually self)>), and note that 
__class__ here is a special variable populated by the compiler.

The first part of this is already documented in super.__doc__ and the 
second part used to be documented:

https://docs.python.org/3.1/library/functions.html?#super

and I don't think there's any harm in adding it back in.

2. Link to the essay on how the MRO is calculated.

https://www.python.org/download/releases/2.3/mro/

> What matters is the __mro__ attribute of the first argument. It matters 
because that is how the MRO actually is searched.

I'm sorry to say that you're wrong here. super() looks at the MRO of the type of the object (the second argument) (*). It has to do that in order to support diamonds. Consider a diamond like

  D
 / \
B   C
 \ /
  A

(with A as common base class). Now super(B, D()).attr will look in the MRO of D (which is D, B, C, A) and therefore delegate to C.attr. In this case, C does not even appear in the MRO of B.

(*) To be pedantic: in the special case that the second argument is a type itself, it looks at the MRO of the second argument.

And this last comment is precisely the kind of information which should be explained in the super() docs.

@Jeroen Demeyer

> I'm sorry to say that you're wrong here. super() looks at the MRO of the type of the object (the second argument) (*).

Exactly! It is funny because I was about to open the same issue this weekend. The documentation of super() is wrong here.

@Steven D'Aprano

> What matters is the __mro__ attribute of the first argument. It matters 
because that is how the MRO actually is searched.

By the way, if it was true (it is not), then what did you think was the purpose of the second parameter of super(type, obj-or-type)?

Please make a concrete proposal (a PR or somesuch). That will make it much easier to determine whether a particular bit of word-smithing is a improvement.

Of the issues discussed so far, these are the most promising:
* Document how the zero argument form of super() infers its arguments.
* Be clear that it is the mro of type(self) that determines the search path and that the role of super() is to the next in the mro after the current class.

> I'm sorry to say that you're wrong here. 

I'm happy to be corrected.

It is fair to say I failed to take the multiple inheritance case into 
account. Clearly super can't *only* look at the MRO of the first object 
since that will miss the multiple inheritance case, as you point out. 
Thank you.

But neither can it *only* look at the MRO of the second class, because 
that would restart the search at the top of the hierarchy; also if 
type(second argument) was the only thing that mattered, that would make 
the first argument redundant and pointless.

> > What matters is the __mro__ attribute of the first argument. It matters 
> because that is how the MRO actually is searched.
> 
> By the way, if it was true (it is not), 

Yes, I see that now.

> then what did you think was 
> the purpose of the second parameter of super(type, obj-or-type)?

Given super(T, obj).method the method object is bound to instance obj; 
how else could it return a bound method instead of an unbound one?

Some of the problems brought up here (which sibling or subclass, and which parameter’s MRO) also came up a few years ago in Issue 23674.

@Steven D'Aprano

> But neither can it *only* look at the MRO of the second class, because that would restart the search at the top of the hierarchy; also if type(second argument) was the only thing that mattered, that would make the first argument redundant and pointless.

Yes of course, but nobody states the opposite. Both parameters are necessary super(type1, obj-or-type2):

— the first one for getting the start class in the MRO, which is the class following type1 in the MRO;
— the second one for getting the MRO, which is type(obj).__mro__ if it is an instance of type1 or type2.__mro__ if it is a subclass of type1, and for binding the function into a method in a super(type1, obj).function expression (as you correctly said in your last message).

Guido's Python equivalent implementation of super() given in his 2002 paper Unifying types and classes in Python 2.2 is very informative:
https://www.python.org/download/releases/2.2.3/descrintro/#cooperation

All this confusion around super() shows that there is room for improvement in the documentation.

@Raymond Hettinger

> Please make a concrete proposal (a PR or somesuch).

I will try to make a PR this weekend if I find some time.

> Some of the problems brought up here (which sibling or subclass, and which parameter’s MRO) also came up a few years ago in Issue 23674.

Indeed. I would actually say that these two issues are duplicates of each other.

Yes, issue 23674 seems to be at least a partial duplicate.  We should just get this fixed and close both issues.

Ideally, the text can also be made more compact.  Having eight paragraphs sends an implicit message that this is too complex to understand and that it should be avoided.

One essential goal is that we need to overcome strong preconceptions about about what super() might do versus what it actually does.  Those with experience of super() in other languages tend to presume that it would be a keyword rather than a builtin type that needs to be instantiated, that it can only work within a class, that it has lower computational overhead than it does, that it only calls parents rather than siblings, that search order is controlled by the method using super() rather than the child instance, that operators will work, or that the search order is something is something other than the C3 algorithm.

The core challenge of documenting super() is that it is so different from what a person might imagine.

On Sat, Jun 08, 2019 at 09:14:18PM +0000, Raymond Hettinger wrote:

> Ideally, the text can also be made more compact.  Having eight 
> paragraphs sends an implicit message that this is too complex to 
> understand and that it should be avoided.

But it is complex -- as you point out, there are *at least* seven 
potential misapprehensions here.

super does a lot: there are at least four ways to call it (zero-argument 
form, unbound super, class+instance and class+subclass). And as the old 
"super considered harmful" versus "super considered super" debate shows, 
there are pitfalls in multiple inheritance that people run into.

The challenge here is that for 95% of cases (plucking a number from thin 
air) using super is simple: just follow the example in the docs. But the 
remaining troublesome cases can be very troublesome, and people running 
into those cases do need to understand the gory details.

> One essential goal is that we need to overcome strong preconceptions 
> about about what super() might do versus what it actually does.
[...]

Here's two more that you missed:

- that you should call super with super(type(self), self);

- that unbound super objects dispatch to unbound methods.

At some point or another over the last decade I've misunderstood super 
to do almost all of those things. As this thread shows, I thought I had 
understood it but I still misunderstood the role of the class calling 
super.

The method says who you will use some recursion.

I've add PR 15564 to clarify some known ambiguities for the super() docs including a specific example "what it actually does".