classification
Title: The repr of BoundMethod objects sometimes incorrectly identifies the bound function
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, Steven.Barker, benjamin.peterson, python-dev
Priority: normal Keywords: patch

Created on 2014-04-30 02:26 by Steven.Barker, last changed 2014-08-20 23:42 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
method_repr.diff Steven.Barker, 2014-05-09 20:15 review
method_repr_tests.diff Steven.Barker, 2014-08-20 06:56 Tests for the new repr for bound methods, and a fix for test_defaultdict
Messages (6)
msg217566 - (view) Author: Steven Barker (Steven.Barker) * Date: 2014-04-30 02:26
The "repr" of bound method objects can be misleading in certain situations. The repr is always is of the format:

    <bound method x.y of <object>>

But "x" is often incorrect.

Here are some examples where the current code gets it wrong:

    # inherited method
    class Base(object):
        def foo(self):
            pass

    class Derived(Base):
        pass

    print(Derived().foo)
    # not too bad, prints "<bound method Derived.foo of <__main__.Derived object at 0xXXXX>>"
    # it probably should say "Base.foo" instead of "Derived.foo", but at least they're two names for the same function

    # however, an override and super() it gets very bad
    class Derived2(Base):
        def foo(self):
            pass

    print(super(Derived2, Derived2()).foo)
    # totally wrong, prints "<bound method Dervied2.foo of __main__.derived2 object at 0xXXXX>>"
    # but it actually *is* Base.foo bound to a Derived2 instance!

    # bound class methods:
    class Test(object):
        @classmethod
        def foo(cls):
            pass

    print(Test.foo)
    # wrong, prints <bound method type.foo of <class '__main__.Test'>>

I suggest that rather than trying to assemble the "x.y" pair by from "__self__.__class__" and "__func__.__name__", the BoundMethod should just use the "__func__.__qualname__". In each of the cases above, the function's location would be correctly located this way.

I came across this bug while investigating a confusing (to me) issue with metaclasses and inheritance. The misleading "repr" output made it much harder to figure out that my expectations were wrong. Here's a simplified example of how it led me astray:

    class A(object):
        @classmethod
        def foo(cls):
            return "classmethod from A"

    class BMeta(type):
        def foo(cls):
            return "instancemethod from BMeta"

    class B(A, metaclass=BMeta):
        pass

    print(B.foo()) # surprisingly (to me) prints "classmethod from A"
    print(B.foo)   # incorrectly prints "<bound method BMeta.foo of <class __main__.B>>"

It is presumably not a bug that inherited class methods take precedence over instance methods from a metaclass (though it was surprising to me at the time). The repr of the bound method though, suggested exactly the opposite. Given that it gets many more common situations wrong as well, I think that the repr should be fixed.

The relevant code appears to be in the method_repr function in Objects/Classobject.c .
msg218196 - (view) Author: Steven Barker (Steven.Barker) * Date: 2014-05-09 20:15
Here's a patch that changes the behavior of method_repr in Objects/classobject.c . It first tries to use __func__.__qualname__, then tries __func__.__name__ as a fallback and finally uses "?" if neither of those attributes are available.

I'm not sure if the __name__ fallback is tested (as it seems that pretty much all callables have __qualname__ these days). The last "?" case actually does get tested by Lib/test/test_descr.py which creates a messed up method with classmethod(1).__get__(1) (which oddly does not raise an error immediately upon creation, but rather only when it is called).

I've not written C in several years, so please let me know if you see I've done something obviously wrong, or any places the patch could be improved. It is mostly a copy-and-paste of existing code with a few modifications and deletions, so hopefully I can't have messed up anything too badly!

I'm currently ignoring a comment in the code that says we "shouldn't use repr()/%R" to format __self__. I don't really understand that, so I've stick with the existing behavior on that front. If that is something that should change, I'd be happy to try reworking it in some other way, just let me know what the concern is.

Here are some examples of the new repr output in a build with the patch:

>>> class A():
...   def foo(self):
...     pass
...
>>> class B(A):
...   def foo(self):
...     pass
...
>>> class C(A):
...   pass
...
>>> class D():
...   @classmethod
...   def bar():
...     pass
...
>>> A().foo
<bound method A.foo of <__main__.A object at 0x02267508>>
>>> B().foo
<bound method B.foo of <__main__.B object at 0x02267578>>
>>> C().foo
<bound method A.foo of <__main__.C object at 0x02267658>>
>>> super(B, B()).foo
<bound method A.foo of <__main__.B object at 0x022676C8>>
>>> D.bar
<bound method D.bar of <class '__main__.D'>>
msg218206 - (view) Author: Steven Barker (Steven.Barker) * Date: 2014-05-10 00:35
Ah, I figured out why using %R may be bad. It breaks for the following silly class:

    class C():
        def __repr__(self):
            return repr(self.__repr__) # or use any other bound method

repr(C()) will recurse until the recursion limit is hit, both with and without my patch. If this seems like a real issue, I could probably replace the %R code with a variation on the base case code in PyObject_Repr:

    PyUnicode_FromFormat("<%s object at %p>",
                         v->ob_type->tp_name, v)
msg225538 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2014-08-19 19:03
That seems reasonable. Can you please write tests for the new behavior, though, and also fix test_defaultdict?
msg225555 - (view) Author: Steven Barker (Steven.Barker) * Date: 2014-08-20 06:56
OK, I've written some tests of the new bound method repr functionality, which I'm attaching as a patch against the current tip.

I test the basic repr output, all the cases the old code got wrong (inherited methods, overridden methods, methods called via super, and classmethods) and the strange corner cases that probably won't come up in ordinary code (such as methods manually created from callables that don't have __name__ or __qualname__ attributes).

I've also fixed the defaultdict test that was relying upon the old repr output. I don't believe there are any other places in the standard library or tests where a bound method's repr is examined.

My patch adds the tests in a new file, Lib/test/test_bound_method_repr.py. I looked to see if there was an existing file that tested similar behavior, but none that I found really seemed appropriate. If I overlooked a better place to put the new tests, please let me know and I'll update the test patch.

I'm not very experienced at writing unit tests, so comments and/or criticism is welcomed. I copied parts of the file's structure (such as the test_main() function and if __name__ == "__main__" boilerplate) from other test files, so hopefully I've stayed pretty close to the usual Python test style.
msg225595 - (view) Author: Roundup Robot (python-dev) Date: 2014-08-20 23:42
New changeset 92dcee426014 by Benjamin Peterson in branch 'default':
use __qualname__ to compute bound method repr (closes #21389)
http://hg.python.org/cpython/rev/92dcee426014
History
Date User Action Args
2014-08-20 23:42:12python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg225595

resolution: fixed
stage: patch review -> resolved
2014-08-20 06:56:39Steven.Barkersetfiles: + method_repr_tests.diff

messages: + msg225555
2014-08-19 19:03:37benjamin.petersonsetmessages: + msg225538
2014-08-14 04:58:54berker.peksagsetnosy: + benjamin.peterson

stage: patch review
2014-05-11 11:52:05Arfreversetnosy: + Arfrever
2014-05-10 00:35:05Steven.Barkersetmessages: + msg218206
2014-05-09 20:15:29Steven.Barkersetfiles: + method_repr.diff
keywords: + patch
messages: + msg218196

versions: + Python 3.5
2014-04-30 02:26:41Steven.Barkercreate