classification
Title: inspect.isgeneratorfunction fails on hand-created methods
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: jdemeyer, ncoghlan, rhettinger, terry.reedy
Priority: normal Keywords: patch

Created on 2018-04-11 08:59 by jdemeyer, last changed 2018-10-09 13:53 by jdemeyer.

Pull Requests
URL Status Linked Edit
PR 6448 open jdemeyer, 2018-04-11 09:35
Messages (10)
msg315187 - (view) Author: Jeroen Demeyer (jdemeyer) * Date: 2018-04-11 08:59
The inspect functions isgeneratorfunction, iscoroutinefunction, isasyncgenfunction can fail on methods that do not have a __code__ attribute:

>>> from types import MethodType
>>> class Callable:
...     def __call__(self, *args):
...         return args
>>> m = MethodType(Callable(), 42)
>>> m()
(42,)
>>> import inspect
>>> inspect.iscoroutinefunction(m)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.6/inspect.py", line 186, in iscoroutinefunction
    object.__code__.co_flags & CO_COROUTINE)
AttributeError: 'Callable' object has no attribute '__code__'

This was discovered while working on PEP 575, but it is really an independent issue that should be fixed anyway.
msg315272 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-04-13 22:46
Nick and Raymond, I added both of you as nosy because, among other reasons, you commented on the latest version of PEP575.

I agree that there is a bug in current CPython, in that the is* functions should return, not raise, but I am not sure where is it, and hence if the PR is the right fix.  I am wondering if, instead, the bug is in m, the object returned by MethodType, or in attribute lookup thereupon.

MethodType is the type of bound user-defined (Python-coded) functions and ismethod is true for instances thereof.  Consider 4 other bound methods, two 'normal'(callable is method of instance class) and two, like Jeroen's, 'odd' (callable not related to int).

>>> cm = Callable().__call__
>>> cm.__code__
<code object __call__ at 0x00000202C1C53780, file "<pyshell#5>", line 2>
>>> cm.__name__
'__call__'

>>> cm2 = MethodType(Callable.__call__, Callable())
>>> cm2.__code__
<code object __call__ at 0x00000202C1C53780, file "<pyshell#5>", line 2>
>>> cm2.__name__
'__call__'

>>> m2 = MethodType(Callable.__call__, 42)
>>> m2.__code__
<code object __call__ at 0x00000202C1C53780, file "<pyshell#5>", line 2>
>>> m2.__name__
'__call__'
>>> m2()
()

>>> m3 = MethodType(Callable().__call__, 42)
>>> m3.__code__
<code object __call__ at 0x00000202C1C53780, file "<pyshell#5>", line 2>
>>> m3.__name__
'__call__'
>>> m3()
(42,)

>>> m = MethodType(Callable(), 42)
>>> m.__code__
... AttributeError: 'Callable' object has no attribute '__code__'
>>> m.__name__
... AttributeError: 'Callable' object has no attribute '__name__'
>>> m()
(42,)

They all have the same attributes exposed by dir(), which omits both'__name__', promised in the docstring for ismethod*, and '__code__', assumed by the is--- functions under discussion.

>>> dir(cm) == dir(cm2) == dir(m) == dir(m2) == dir(m3)
True
>>> ('__code__' in dir(cm)) or ('__name__' in dir(cm))
False

However, accessing those names anyway, as (hidden) attributes, works for all but m.  Should doing so also work for m?  (If not, the ismethod docstring should not 'guarantee' .__name__.)

* "    Instance method objects provide these attributes:
        __doc__         documentation string
        __name__        name with which this method was defined
..."
msg315284 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-04-14 05:04
The inspect functions throwing an exception when handed a method wrapping a non-function callable instead of returning False is a definite bug.

The behaviour of MethodType when handed a non-function callable is cryptic, but not actually a bug: the problem is that it expects to be able to delegate accesses to __name__ and __code__ to the underlying callable.

A docs bug to clarify exactly which accesses are going to get delegated would make sense (and perhaps make MethodType.__dir__ a bit smarter about that delegation), but it's a separate issue from the one Jeroen has reported here.
msg315287 - (view) Author: Jeroen Demeyer (jdemeyer) * Date: 2018-04-14 13:51
> I am wondering if, instead, the bug is in m, the object returned by MethodType, or in attribute lookup thereupon.

What would you expect m.__code__ to return then? Methods support arbitrary callables and certainly not all callables have a meaningful __code__.
msg315288 - (view) Author: Jeroen Demeyer (jdemeyer) * Date: 2018-04-14 13:59
The only attributes that a method is guaranteed to have are __func__ and __self__ (which are the arguments passed to the MethodType constructor).

All other attributes are looked up through __func__ by the C version of the following Python code:

def __getattr__(self, attr):
    return getattr(self.__func__, attr)
msg315302 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-04-14 19:07
I would like python_coded_callable.__code__ to be the code object executed when python_coded_callable is called, just as expected by the isxyz author(s).  It has to exist somewhere.  Methods m and m3 both return 42 when called, and both have the same code object.

>>> m3.__code__
<code object __call__ at 0x000001431C797E40, file "F:\Python\a\tem2.py", line 3>
>>> m3.__func__.__code__
<code object __call__ at 0x000001431C797E40, file "F:\Python\a\tem2.py", line 3>
>>> m.__func__.__call__.__code__
<code object __call__ at 0x000001431C797E40, file "F:\Python\a\tem2.py", line 3>

The fact that m requires an additional level of indirection is an implementation detail that I don't think necessarily has to involve users.
msg315305 - (view) Author: Jeroen Demeyer (jdemeyer) * Date: 2018-04-14 20:47
> I would like python_coded_callable.__code__ to be the code object executed when python_coded_callable is called

First of all, that doesn't answer the question of what to do with non-Python coded callables where there is no __code__ object at all.

Second, are you *really* sure that you want that? That would mean adding a __code__ attribute to all callable Python classes or adding a __code__ descriptor to "type".
msg315312 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-04-15 06:46
Terry, if you'd like to continue that discussion, please open a new enhancement request for 3.8+ against the inspect module asking for the affected introspection functions to recursively search for relevant attributes, the same way `inspect.signature` digs inside different object types to look for signature information.

However, the fact the inspect module doesn't implement that search for relevant attributes today is *not* a bug - it's a past design decision that could potentially stand to be revisited.

By contrast, the fact the affected functions throw AttributeError instead of returning False for a missing attribute *is* a bug.
msg315438 - (view) Author: Jeroen Demeyer (jdemeyer) * Date: 2018-04-18 07:25
Can we please go back to the original issue? If you think that __code__ should be an alias for __call__.__code__, that is a different issue.

On https://github.com/python/cpython/pull/6448#issuecomment-381507329 I posted an alternative solution for the problem. Is either the original PR or the new variant acceptable?
msg327401 - (view) Author: Jeroen Demeyer (jdemeyer) * Date: 2018-10-09 13:53
Can somebody please review PR 6448?
History
Date User Action Args
2018-10-09 13:53:19jdemeyersetmessages: + msg327401
2018-04-18 07:25:22jdemeyersetmessages: + msg315438
2018-04-15 06:46:58ncoghlansetmessages: + msg315312
2018-04-14 20:47:00jdemeyersetmessages: + msg315305
2018-04-14 19:07:52terry.reedysetmessages: + msg315302
2018-04-14 13:59:05jdemeyersetmessages: + msg315288
2018-04-14 13:51:33jdemeyersetmessages: + msg315287
2018-04-14 05:04:54ncoghlansetmessages: + msg315284
2018-04-13 22:46:48terry.reedysetnosy: + rhettinger, terry.reedy, ncoghlan
messages: + msg315272
2018-04-11 09:35:39jdemeyersetkeywords: + patch
stage: patch review
pull_requests: + pull_request6144
2018-04-11 08:59:53jdemeyercreate