This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: inspect.getsource(obj.foo) fails when foo is an injected method constructed from another method
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eric.snow, furkanonder, mtahmed, r.david.murray, serhiy.storchaka, terry.reedy, ueg1990, vajrasky
Priority: normal Keywords: patch

Created on 2013-12-11 23:27 by mtahmed, last changed 2022-04-11 14:57 by admin.

Files
File name Uploaded Description Edit
reproducer.py mtahmed, 2013-12-11 23:27 reproducer
issue_19956.patch ueg1990, 2015-04-13 16:47 review
issue_19956_1.patch ueg1990, 2015-04-14 22:41 review
Pull Requests
URL Status Linked Edit
PR 20025 closed furkanonder, 2020-05-10 17:05
Messages (11)
msg205942 - (view) Author: Muhammad Tauqir Ahmad (mtahmed) Date: 2013-12-11 23:27
If a method `foo` of object instance `obj` is injected into it using a method from a different object instance, `inspect.getsource(obj.foo)` fails with the error message:

TypeError: <bound method Foo.say of <__main__.Foo object at 0x7fd348662cd0>> is not a module, class, method, function, traceback, frame, or code object

in inspect.py:getfile()

What basically is happening is that if you have `obj1`, `obj2` and `obj2` has method `foo`, setting `obj1.foo = types.MethodType(obj2.foo, obj1)` succeeds but is "double-bound-method" if that's a term. So during `getsource()`, it fails because `obj1.foo.__func__` is a method not a function as is expected.

Possible solutions:
1. Error message should be more clear if this is the intended behavior - right now it claims it's not a method,function etc. when it is indeed a method.
2. MethodType() should fail if the first argument is not a function.
3. inspect.getsource() should recursively keep "unpacking" till it finds an object that it can get the source for. 

Reproducer attached.
msg205948 - (view) Author: Vajrasky Kok (vajrasky) * Date: 2013-12-12 09:58
FYI, if you change:

setattr(b, 'say', types.MethodType(f.say, b))

to:

setattr(b, 'say', types.MethodType(Foo.say, b))

it will print the source correctly.
msg205972 - (view) Author: Muhammad Tauqir Ahmad (mtahmed) Date: 2013-12-12 18:33
Yes I understand your change and other possible changes will fix the reproducer. I am already using a different workaround in my code.

The issue is about `inspect.getsource()` having incorrect behavior (or at least inaccurate error message) and MethodType able to be constructed from another method leading to strange undocumented behavior (as far as I can tell).

I do not know what the correct resolution is to this issue which is why I posted this here so someone can suggest/approve a resolution and I can submit a patch once something is decided.
msg206164 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-12-14 03:59
I do not think you have exactly identified a bug, certainly not one we would fix in an existing release. The behavior in more of an unintended consequence of separate decisions resulting from an unanticipated usage. I am not sure what, if anything, should be done. Changing the exception message could still be done in 3.4, but not making getsource recursive for bound methods.
msg240653 - (view) Author: Usman Ehtesham Gul (ueg1990) * Date: 2015-04-13 16:47
After discussing with Eric Snow, this case scenario is an edge case. The assumption in the inspect module is that __func__ for a MethodType object is a function. The MethodType should be used for functions and not methods.

Patch attached for this.

This needs to be documented in the docs (separate ticket will be opened for that).
msg240766 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-04-13 21:34
Thanks for the patch.  I've made some review comments on the tests.
msg241049 - (view) Author: Usman Ehtesham Gul (ueg1990) * Date: 2015-04-14 22:41
Made changes based on David Murray's review comments including adding unit test on getfile.
msg296284 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-06-18 18:14
Any review of the revised patch?
msg368591 - (view) Author: Furkan Onder (furkanonder) * Date: 2020-05-10 18:23
PR has been sent.
msg368647 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-05-11 20:48
I am working on an explanation of why I paused the PR.
msg369174 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-05-18 04:30
Here is minimal reproducing code.

import types
import inspect

class A:
    def say(self): print("A.say")
a = A()

class B: pass    
b = B()
b.say = types.MethodType(a.say, b)

Let us examine MethodType first.  Calling 'b.say()' asks the previously neglected question: Is b.say a proper, sane callable?  I claim not. With 3.9.0a6, the call fails with "TypeError: say() takes 1 positional argument but 2 were given".  b.say() calls a.say(b), which calls A.say(a, b).  If A.say took another parameter, such as 'name', 'b.say()' might work, but only by accident.

Here is another buggy use of MethodType, 
b.A = types.MethodType(A, b)
b.A()
# TypeError: A() takes no arguments
Again, given 'def __init__(self, something)', b.A() might work, but only by accident.

types.MethodType is an example of "[This module] defines names for some object types that are used by the standard Python interpreter, but not exposed as builtins like int or str are."  The names are mainly intended to be used for isinstance checks and rarely, if at all, for object creation.

The MethodType entry lack a signature and only says "The type of methods of user-defined class instances."  Its docstring, "method(function, instance)\n\nCreate a bound instance method object." does have a signature.  However, 'function' must be a function compatible with being passed 'instance' as the first argument.  This is false for both A and a.say; both result in buggy 'callables'.

MethodType checks that its first argument, assigned to the .__func__ attribute, is callable (has a '.__call__' attribute) but apparently does not check further.

As for getsource.  For b.A and b.say, it raises "TypeError: module, class, method, function, traceback, frame, or code object was expected, got {type}", where type is 'type' and 'method' respectively.  For both, the message is slightly confusing in that the function got something on the list (type=class).  For both, getsource could be patched to work with the buggy inputs.  I the latter is a bad idea.  Built-in functions usually fail with buggy inputs.  We should either improve the error message for methods or just close this.
History
Date User Action Args
2022-04-11 14:57:55adminsetgithub: 64155
2020-05-18 04:30:33terry.reedysetnosy: + serhiy.storchaka
messages: + msg369174
2020-05-11 20:48:11terry.reedysetmessages: + msg368647
versions: + Python 3.9, - Python 3.5
2020-05-10 18:23:39furkanondersetmessages: + msg368591
2020-05-10 17:05:51furkanondersetnosy: + furkanonder

pull_requests: + pull_request19335
stage: needs patch -> patch review
2017-06-18 18:14:17terry.reedysetmessages: + msg296284
2015-04-14 22:41:42ueg1990setfiles: + issue_19956_1.patch

messages: + msg241049
2015-04-13 21:34:59r.david.murraysetnosy: + r.david.murray
messages: + msg240766
2015-04-13 16:47:43ueg1990setfiles: + issue_19956.patch

nosy: + eric.snow, ueg1990
messages: + msg240653

keywords: + patch
2013-12-14 03:59:09terry.reedysetversions: + Python 3.5, - Python 2.6, Python 3.1, Python 2.7, Python 3.2, Python 3.3
nosy: + terry.reedy

messages: + msg206164

type: behavior -> enhancement
stage: needs patch
2013-12-12 18:33:35mtahmedsetmessages: + msg205972
2013-12-12 09:58:03vajraskysetnosy: + vajrasky
messages: + msg205948
2013-12-11 23:27:20mtahmedcreate