This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Incorrect description of descriptor invocation in Python Language Reference
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Justin.Eldridge, docs@python, iritkatriel, r.david.murray, rhettinger
Priority: normal Keywords:

Created on 2015-01-25 17:16 by Justin.Eldridge, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (5)
msg234676 - (view) Author: Justin Eldridge (Justin.Eldridge) Date: 2015-01-25 17:16
The section titled "Invoking Descriptors" in the Python Language Reference [1]
says:

    Class Binding
        If binding to a new-style class, A.x is transformed into the call: 
        A.__dict__['x'].__get__(None, A).

This suggests that __get__ is looked up on the instance of x, when I believe
that it is actually looked up on the type. That is, it's my understanding that
A.x invokes:

    type(A.__dict__['x']).__get__(A.__dict__['x'], None, Foo))

Here's some Python 3.4 code demonstrating this:

    class A:
        pass

    class Descriptor:
        def __get__(self, obj, cls):
            print("Getting!")
            
    A.x = Descriptor()

    def replacement_get(obj, cls):
        print("This is a replacement.")
        
    A.__dict__['x'].__get__ = replacement_get

Now, writing:

    >>> A.x
    Getting!

    >>> A.__dict__['x'].__get__(None, A)
    This is a replacement!

    >>> type(A.__dict__['x']).__get__(A.__dict__['x'], None, A)
    Getting!

The documentation makes a similar statement about instance binding that also
appears to be incorrect. This is the case in all versions of the document I
could find.

What I believe to be the actual behavior is implied by a later section in the
same document, which states that the implicit invocation of special methods is
only guaranteed to work correctly if the special method is defined on the type,
not the instance.  This suggests that the statements in "Invoking Descriptors"
aren't quite correct, and while the true behavior is a little more verbose, I
think it would be worthwhile to update the documentation so as to avoid
confusion.

[1]: https://docs.python.org/2/reference/datamodel.html#invoking-descriptors
msg234684 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-01-25 20:28
I believe that you are correct: special methods are looked up on the type, and this is assumed implicitly in the Class Binding description. (This was not 100% true in python2, but it is in current python3.)  But the Class Binding description is correct, since if the __get__ method is *not* defined on the type (of the descriptor), the descriptor instance itself is returned (so explicitly calling type in the "equivalent expression" would be wrong).

This is one of the most complex topics to describe in Python (I still don't have it solid in my head and I've been working with Python for years now).  If we can come up with clearer wording that is good, but we've tried several times already to improve it :(

I don't know what you are referring to for the 'instance binding' that makes the same 'mistake', but I suspect it is also covered by the "special methods are looked up on the type" rule.
msg234763 - (view) Author: Justin Eldridge (Justin.Eldridge) Date: 2015-01-26 17:54
Ah, I see how writing a description of this which is both concise and precise
would be difficult, especially for Python 2.

> But the Class Binding description is correct, since if the __get__ method is
> *not* defined on the type (of the descriptor), the descriptor instance itself
> is returned (so explicitly calling type in the "equivalent expression" would be
> wrong)

I see, but couldn't this also be held against the current "equivalent"?  That
is, saying A.x is equivalent to A.__dict__['x'].__get__(None, A) is not stricly
true when __get__ isn't defined on type(x).

I think I see now why this is difficult to document cleanly, and the "Special
method lookup" section of the documentation does a good job of explaining this.
The issue isn't exclusive to descriptors. It affects, for example, the
documentation on rich comparison operators, which says that x == y invokes
x.__eq__(y), when this hasn't quite been true since old-style classes.

So saying x == y is equivalent to x.__eq__(y) isn't really correct, and saying
that it is equivalent to type(x).__eq__(x,y) isn't quite right either, as
implicit invocation may bypass the metaclass's __getattribute__. The latter,
however, seems "less wrong". Is there a reason that the former is preferred by
the documentation?
msg407605 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-03 21:48
See also issue20751.
msg407624 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-12-04 01:54
I think we can leave this as-is.  It does a reasonable job of communicating where the descriptor is found and the arguments used when it is called.

Marking this as out of date because later in the howto guide there is a precise pure python equivalent for the lookup and invocation steps.
History
Date User Action Args
2022-04-11 14:58:12adminsetgithub: 67506
2021-12-04 01:54:16rhettingersetstatus: open -> closed
resolution: out of date
messages: + msg407624

stage: resolved
2021-12-03 21:48:31iritkatrielsetnosy: + iritkatriel, rhettinger
messages: + msg407605
2015-01-26 17:54:08Justin.Eldridgesetmessages: + msg234763
2015-01-25 20:28:10r.david.murraysetnosy: + r.david.murray

messages: + msg234684
versions: - Python 3.2, Python 3.3, Python 3.6
2015-01-25 17:16:03Justin.Eldridgecreate