This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Harmonize descriptor protocol documentation: direct call, super binding with Descriptor Howto docs
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: not a bug
Dependencies: 12077 Superseder:
Assigned To: rhettinger Nosy List: Arthur-Milchior, docs@python, martin.panter, rhettinger, zuo
Priority: normal Keywords: patch

Created on 2014-02-23 22:13 by zuo, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 29909 merged rhettinger, 2021-12-03 23:47
PR 29980 merged rhettinger, 2021-12-08 03:18
PR 30271 closed Arthur-Milchior, 2021-12-26 21:10
Messages (13)
msg212035 - (view) Author: Jan Kaliszewski (zuo) Date: 2014-02-23 22:13
1. One misleading detail in the descriptor protocol documentation for super bindings is that the following fragment of the http://docs.python.org/reference/datamodel.html#invoking-descriptors page:

"""
Super Binding
    If a is an instance of super, then the binding super(B, obj).m() searches obj.__class__.__mro__ for the base class A immediately preceding B and then invokes the descriptor with the call: A.__dict__['m'].__get__(obj, obj.__class__).
"""

...introduces the method *call* (".m()") which AFAIK has nothing to do with the actual matter of the description (attribute resolution).

Also, the "If *a* is an instance of super" fragment is strange, as *a* is not used in the following sentences at all.

I believe the description should be:

"""
Super Binding
    If binding to a super instance, super(B, obj).x searches obj.__class__.__mro__ for the base class A immediately preceding B and then invokes the descriptor with the call: A.__dict__['x'].__get__(obj, obj.__class__).
"""

(using 'x' as the attribute name, as for the other kinds of binding).

***

2. Also, in some earlier fragment of the same page:

"""
Direct Call
    The simplest and least common call is when user code directly invokes a descriptor method: x.__get__(a).
"""

The call x.__get__(a) without the second argument seems to be wrong if  __get__ is implemented according to the specification "object.__get__(self, instance, owner)" from the same documentation page.
msg237709 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2015-03-09 21:37
Who is best placed to comment on the suggested docs changes given in msg212035?
msg294023 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2017-05-20 09:35
Lower-case “a” is defined at the top of the list: “The starting point . . . is ‘a.x’.” The last entry may fit in better if it was written “If binding to an instance of ‘super’ ”.

The problem with the m() call is also mentioned in Issue 25777, about the descriptor how-to.

Issue 12077 seems to be largely about the __get__ signature.
msg407617 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-12-03 23:39
Regarding comment #1, The wording is correct and there was a reason for using a method.  While super() can be used for attribute lookup, use cases are almost entirely dominated by method lookups.  For many users, an attribute lookup with super() is unconventional, weird, hard to grok, awkward to demonstrate, and not well motivated by the way super() is actually used.

    ##############################################################
    # Demonstration code for the example in text

    class A:
        def m(self):
            return 42

    class B(A):
        def m(obj):
            return super(B, obj).m()

    >>> b = B()
    >>> b.m()                             # Normal invocation 
    42
    >>> A.__dict__['m'].__get__(b, B)()   # Equivalent call
    42

That said, I will switch it to an attribute lookup for consistency with the other examples in the section and with the current version of the DescriptorHowto.


Regarding comment #2, the objtype argument is optional as shown in all of the examples.  The call from object.__getattribute__() always passes in both parameters, even though only the first is required.
  
    ################################################################
    # Demonstration of __get__() being called with one or two params

    class A:
        def __init__(self, x):
            self.x = x
        def m(self, y):
            return self.x * y

    >>> a = A(10)
    >>> a.m(5)
    50
    >>> vars(A)['m'].__get__(a)(5)     # objtype is not required
    50
    >>> vars(A)['m'].__get__(a, A)(5)  # objtype may be used
    50


    ################################################################
    # Demonstration of object.__getattribute__ supplying both args

    class Desc:
        def __get__(self, *args):
            return args
      
    class B:
        z = Desc()
     
    >>> b = B()
    >>> b.z
    (<__main__.B object at 0x109156110>, <class '__main__.B'>)
msg407627 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-12-04 02:37
New changeset 135ecc3492cee259090fd4aaed9056c130cd2eba by Raymond Hettinger in branch 'main':
bpo-20751:  Replace method example with attribute example, matching the descriptor howto (GH-29909)
https://github.com/python/cpython/commit/135ecc3492cee259090fd4aaed9056c130cd2eba
msg407965 - (view) Author: Jan Kaliszewski (zuo) Date: 2021-12-07 20:55
So the current (after the aforementioned commit) form of the description is:

   A dotted lookup such as ``super(A, a).x`` searches
   ``obj.__class__.__mro__`` for a base class ``B`` following ``A`` and then
   returns ``B.__dict__['x'].__get__(a, A)``.  If not a descriptor, ``x`` is
   returned unchanged.

I guess here ``obj`` was supposed to be ``a``.

But is the description correct when it comes to what class is used where?
I.e., shouldn't it be rather something along the lines of the following:

   A dotted lookup such as ``super(A, obj).x`` (where ``obj`` is an
   instance of ``A`` of some other subclass of ``A``) searches
   ``A.__mro__`` for a base class ``B`` whose `__dict__` contains name
   ``"x"`` and then returns ``B.__dict__['x'].__get__(obj, type(obj))``.
   If ``B.__dict__['x']`` is not a descriptor, it is returned unchanged.

***

Ad my comment #2 -- yes, it became groundless with time... Minor explanation: when I reported this issue in 2015, the signature of `object.__get__` was documented just as "__get__(self, instance, owner)" (see: https://docs.python.org/3.5/reference/datamodel.html#implementing-descriptors); that's why I wrote about an inconsistency.
msg407966 - (view) Author: Jan Kaliszewski (zuo) Date: 2021-12-07 21:01
Sorry, a few mistakes distorted my proposal. It should be:

   A dotted lookup such as ``super(A, obj).x`` (where ``obj`` is an
   instance of ``A`` or of a subclass of ``A``) searches ``A.__mro__``
   for a base class whose `__dict__` contains name ``"x"``, and
   then returns ``B.__dict__['x'].__get__(obj, type(obj))`` (where
   ``B`` is that base class).  If ``B.__dict__['x']`` is not a
   descriptor, it is returned unchanged.
msg407970 - (view) Author: Jan Kaliszewski (zuo) Date: 2021-12-07 21:10
I am very sorry, I just noticed another mistake.

It should be:

   A dotted lookup such as ``super(A, obj).x`` (where ``obj``
   is an instance of ``A`` or of a subclass of ``A``) searches
   ``type(obj).__mro__`` for such a base class ``B`` that follows
   ``A`` and whose :attr:`__dict__` contains the name ``"x"``;
   then ``B.__dict__['x'].__get__(obj, type(obj))`` is returned.
   If not a descriptor, ``B.__dict__['x']`` is returned unchanged.
msg407995 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-12-08 03:38
New changeset 4ccccb1cfc1f30327e76a2d845cc274be56b34b1 by Raymond Hettinger in branch 'main':
bpo-20751: Match variable name to the example. (GH-29980)
https://github.com/python/cpython/commit/4ccccb1cfc1f30327e76a2d845cc274be56b34b1
msg407996 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-12-08 03:44
> I guess here ``obj`` was supposed to be ``a``.

Okay, I updated the variable name to match the rest of the example.

> But is the description correct when it comes to what class is used where?

It looks fine to me.  It is harmonious with the other three entries in the section and every part of the wording is verified in the test code.

> It should be:

Sorry, I disagree with the wordsmithing.  IMO, the additional proposed wording mostly makes it harder to read.  Also, this section is already a bit long. The details on super() are in the main docs for super().  Here we just want to show that super() is one of the four ways to invoke a descriptor.
msg408410 - (view) Author: Jan Kaliszewski (zuo) Date: 2021-12-12 22:16
Sure. But don't you think there should be ``.__get__(a, type(a))`` rather than ``.__get__(a, A)``? Then the whole statement would be true regardless of whether A is the actual type of a, or only a superclass of the type of a.

That would also be more consistent with the second point of the description, i.e., the one about *Instance Binding* (where we have ``type(a).__dict__['x'].__get__(a, type(a))``).

Also, I believe that ``type(a).__mro__`` would be more consistent (than ``a.__class__.mro``) with that point.
msg409204 - (view) Author: Arthur Milchior (Arthur-Milchior) * Date: 2021-12-26 16:15
Shouldn't those change be ported to 3.10 and maybe even earlier version? They are great, but hard to find if you read the current version manual.

I'm just surprised by the term "dotted lookup". The remaining of the documentation mention "attribute access". So I assumed that there was a reason to use "dotted lookup" instead of "attribute access". If there is such a reason, it's not clear right now. Otherwise, if they are equivalent, I'd suggest remaining consistent in the term used.

I must note that "dotted lookup" is also used in the howto about descriptor, but it does not explain the difference with "attribute access".

One last note, I find it a little bit strange that `a` was supposed to be the value on which the access is done, and suddenly it becomes a part of the value.
msg409208 - (view) Author: Arthur Milchior (Arthur-Milchior) * Date: 2021-12-26 16:27
"a base class ``B`` following ``A``" shouldn't it be "the base class"? . After all, there is at most one base class following ``A``.

Also, I find it unclear what means "``x`` is returned unchanged, since in this context ``x`` is not a value which exists by itself.
History
Date User Action Args
2022-04-11 14:57:59adminsetgithub: 64950
2021-12-26 21:10:15Arthur-Milchiorsetpull_requests: + pull_request28486
2021-12-26 16:27:34Arthur-Milchiorsetmessages: + msg409208
2021-12-26 16:15:58Arthur-Milchiorsetnosy: + Arthur-Milchior
messages: + msg409204
2021-12-12 22:16:59zuosetmessages: + msg408410
2021-12-08 03:44:46rhettingersetstatus: open -> closed

messages: + msg407996
stage: patch review -> resolved
2021-12-08 03:38:39rhettingersetmessages: + msg407995
2021-12-08 03:18:11rhettingersetstage: resolved -> patch review
pull_requests: + pull_request28206
2021-12-07 21:10:56zuosetmessages: + msg407970
2021-12-07 21:01:55zuosetmessages: + msg407966
2021-12-07 20:55:49zuosetstatus: closed -> open

messages: + msg407965
2021-12-04 02:39:44rhettingersetstatus: open -> closed
stage: patch review -> resolved
2021-12-04 02:37:15rhettingersetmessages: + msg407627
2021-12-03 23:47:23rhettingersetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request28134
2021-12-03 23:39:41rhettingersetresolution: not a bug
messages: + msg407617
2021-12-03 16:19:24rhettingersettitle: Misleading descriptor protocol documentation: direct call, super binding -> Harmonize descriptor protocol documentation: direct call, super binding with Descriptor Howto docs
2021-12-03 16:17:57rhettingersetassignee: docs@python -> rhettinger
2021-12-03 14:18:05iritkatrielsettype: behavior
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 2.7, Python 3.5, Python 3.6, Python 3.7
2017-05-20 13:36:56BreamoreBoysetnosy: - BreamoreBoy
2017-05-20 09:35:51martin.pantersetdependencies: + Harmonizing descriptor protocol documentation

title: Misleading examples in the descriptor protocol documentation -> Misleading descriptor protocol documentation: direct call, super binding
nosy: + rhettinger, martin.panter
versions: + Python 2.7, Python 3.6, Python 3.7, - Python 3.4
messages: + msg294023
stage: needs patch
2015-03-09 21:37:59BreamoreBoysetnosy: + BreamoreBoy

messages: + msg237709
versions: + Python 3.5, - Python 3.1, Python 2.7, Python 3.2, Python 3.3
2014-02-23 22:51:39zuosettitle: Misleading examples indDescriptor protocol documentation -> Misleading examples in the descriptor protocol documentation
2014-02-23 22:13:56zuocreate