This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: inspect.getsource does not work with decorated functions
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: yselivanov Nosy List: Claudiu.Popa, Eric.Wieser, belopolsky, christian.heimes, gpolo, michele_s, pitrou, python-dev, yselivanov, ysj.ray
Priority: normal Keywords: patch

Created on 2007-07-31 10:16 by michele_s, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
inspect.py.diff gpolo, 2008-02-07 20:10 review
inspect_getsource.patch Claudiu.Popa, 2013-09-29 09:51 review
Messages (12)
msg32574 - (view) Author: Michele Simionato (michele_s) Date: 2007-07-31 10:16
Here is the issue:

$ cat example.py
import functools # I am using Python 2.5

def identity_dec(func):
    def wrapper(*args, **kw):
        return func(*args, **kw)
    return functools.update_wrapper(wrapper, func)

@identity_dec
def example(): 
    pass

>>> import inspect
>>> from example import example
>>> print inspect.getsource(example)
    def wrapper(*args, **kw):
        return func(*args, **kw)

You get the source code of the closure 
and not what would be more meaningful, i.e.
the string

"""
@identity_dec
def example(): 
    pass
"""

Of course one could argue that this is not a bug
(in a sense the inspect module is doing the right
thing) but still it is giving information which
is not very useful.

Looking at the guts of inspect.getsource, one discovers the origin 
of the problem: inspect.findsource is looking at the attribute 
.co_firstlineno of the decorated function code object.
Unfortunately .co_firstlineno is a read-only attribute, otherwise
it would be possibile to change functools.update_wrapper to set it to
the correct line number (i.e. the line where
the undecorated function is defined, -1). So 
I don't think you can fix this in current
Python, but it is something to keep in mind for Python 2.6 and 3.0. It should also manage classmethods/
staticmethods and other decorators not implemented
as closures.
msg57779 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-23 09:13
I'm setting the target to 2.6 and 3.0. Maybe somebody can come up with a
sensible patch.
msg60062 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-01-17 21:29
Rather than devising something specific to the co_firstlineno attribute,
why not have functools.update_wrapper add a "wrapped_func" attribute
pointing to the original function object? That way, each function
inspecting the decorated function would have the opportunity to walk the
decoration chain if it wants.
msg62177 - (view) Author: Guilherme Polo (gpolo) * (Python committer) Date: 2008-02-07 20:10
I am attaching a patch that address this issue.
msg110708 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-19 02:02
I did not test the patch, but if it really addresses the issue, I don't see why such a simple fix would not be applied.  


Guilherme,

Can you add unit tests? Also, the second chunk in your patch is just a style fix which is a distraction for reviewers.
msg110752 - (view) Author: ysj.ray (ysj.ray) Date: 2010-07-19 13:40
I don't think this patch is correct, because we don't know if the closure[0] is the wrapped function, consider the following case:

def fun():
    abc = 1
    def fun2():
        print(abc)
    print(inspect.getsource(fun2))

In this case, the __closure__ of fun2 is [cell(1), ], the patch doesn't work.


I think the behavior of inspect.getsource(arg) is no problem because it indeed gives the right source code of arg, no matter arg is a wrapper function or an original function. michele argues that the result is not meaningful, through in most case it seems reasonably because wrapper functions usually do litter valuable work, but it is not correct for all cases. What if a wrapper function does more valuable work than the wrapped function? In this case should getsouce() give the source code of wrapper function because it's more meaningful? The concept "wrapper" and "wrapped" should have no relation with its source code.

I suggest you assign a special named attribute of all wrapper functions to its wrapped function yourself, something like "wrapped_function", and then you can walk through the wrapper chain to find the real source you want for each wrapper function. Then the stdard library's update_wrapper() and getsource() can remain unchanged.
msg198575 - (view) Author: PCManticore (Claudiu.Popa) * (Python triager) Date: 2013-09-29 09:51
Hello. Attached patch which uses the new inspect.unwrap, added in http://hg.python.org/cpython/rev/2aa6c1e35b8a.
msg227654 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-09-26 21:35
New changeset ad9cc6124a19 by Yury Selivanov in branch 'default':
inspect: Fix getsource() to support decorated functions.
https://hg.python.org/cpython/rev/ad9cc6124a19
msg227655 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2014-09-26 21:36
Thanks for the bug report and patch! Committed to 3.5.
msg322728 - (view) Author: Eric Wieser (Eric.Wieser) * Date: 2018-07-31 06:13
This now leaves `inspect.getsource` inconsistent with `inspect.getsourcefile`:

>>> import inspect
>>> from contextlib import contextmanager

>>> @contextmanager
... def func():
...    yield

>>> inspect.getsource(func)
'@contextmanager\ndef func():\n    yield\n'
>>>inspect.getsourcefile(func)
'C:\\Program Files\\Python 3.5\\lib\\contextlib.py'

Should `getsourcefile` be changed to match?

This is causing numpy/numpy#11639, but it's not clear if this is a bug or by design.
msg322775 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-07-31 14:01
> Should `getsourcefile` be changed to match?

I'd say yes. There's no point in getsourcefile returning the file location of the topmost decorator. Feel free to open a new issue and submit a PR to fix this!
msg322918 - (view) Author: Eric Wieser (Eric.Wieser) * Date: 2018-08-02 06:34
New issue opened at https://bugs.python.org/issue34305, along with a PR linked there.
History
Date User Action Args
2022-04-11 14:56:25adminsetgithub: 45259
2018-08-02 06:34:15Eric.Wiesersetmessages: + msg322918
2018-07-31 14:01:31yselivanovsetmessages: + msg322775
2018-07-31 06:13:44Eric.Wiesersetnosy: + Eric.Wieser
messages: + msg322728
2014-09-26 21:46:22berker.peksagsetstage: patch review -> resolved
2014-09-26 21:36:35yselivanovsetstatus: open -> closed
resolution: fixed
messages: + msg227655
2014-09-26 21:35:37python-devsetnosy: + python-dev
messages: + msg227654
2014-08-25 08:54:16Claudiu.Popasetstage: test needed -> patch review
2014-04-15 18:02:34yselivanovsetassignee: yselivanov
2014-01-28 21:43:59yselivanovsetnosy: + yselivanov

versions: + Python 3.5, - Python 2.7, Python 3.2, Python 3.3, Python 3.4
2013-09-29 09:51:22Claudiu.Popasetfiles: + inspect_getsource.patch

nosy: + Claudiu.Popa
messages: + msg198575

keywords: + patch
2013-08-04 19:58:06belopolskysetassignee: belopolsky -> (no value)
2013-01-27 20:54:43ezio.melottisetversions: + Python 3.3, Python 3.4
2010-07-19 13:40:57ysj.raysetnosy: + ysj.ray
messages: + msg110752
2010-07-19 02:02:28belopolskysetassignee: belopolsky
type: behavior
versions: + Python 2.7, Python 3.2, - Python 2.6, Python 3.0
nosy: + belopolsky

messages: + msg110708
stage: test needed
2008-02-07 20:10:28gpolosetfiles: + inspect.py.diff
nosy: + gpolo
messages: + msg62177
2008-01-17 21:29:40pitrousetnosy: + pitrou
messages: + msg60062
2007-11-23 09:13:58christian.heimessetnosy: + christian.heimes
title: inspect.getsource does not work with decorated functions -> inspect.getsource does not work with decorated functions
messages: + msg57779
versions: + Python 2.6, Python 3.0, - Python 2.5
2007-07-31 10:16:40michele_screate