Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

inspect.getsource does not work with decorated functions #45259

Closed
micheles mannequin opened this issue Jul 31, 2007 · 12 comments
Closed

inspect.getsource does not work with decorated functions #45259

micheles mannequin opened this issue Jul 31, 2007 · 12 comments
Assignees
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@micheles
Copy link
Mannequin

micheles mannequin commented Jul 31, 2007

BPO 1764286
Nosy @abalkin, @pitrou, @tiran, @PCManticore, @1st1, @eric-wieser
Files
  • inspect.py.diff
  • inspect_getsource.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/1st1'
    closed_at = <Date 2014-09-26.21:36:35.226>
    created_at = <Date 2007-07-31.10:16:40.000>
    labels = ['type-bug', 'library']
    title = 'inspect.getsource does not work with decorated functions'
    updated_at = <Date 2018-08-02.06:34:15.069>
    user = 'https://bugs.python.org/micheles'

    bugs.python.org fields:

    activity = <Date 2018-08-02.06:34:15.069>
    actor = 'Eric.Wieser'
    assignee = 'yselivanov'
    closed = True
    closed_date = <Date 2014-09-26.21:36:35.226>
    closer = 'yselivanov'
    components = ['Library (Lib)']
    creation = <Date 2007-07-31.10:16:40.000>
    creator = 'michele_s'
    dependencies = []
    files = ['9384', '31902']
    hgrepos = []
    issue_num = 1764286
    keywords = ['patch']
    message_count = 12.0
    messages = ['32574', '57779', '60062', '62177', '110708', '110752', '198575', '227654', '227655', '322728', '322775', '322918']
    nosy_count = 10.0
    nosy_names = ['michele_s', 'belopolsky', 'pitrou', 'christian.heimes', 'gpolo', 'ysj.ray', 'Claudiu.Popa', 'python-dev', 'yselivanov', 'Eric.Wieser']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue1764286'
    versions = ['Python 3.5']

    @micheles
    Copy link
    Mannequin Author

    micheles mannequin commented Jul 31, 2007

    Here is the issue:

    $ cat example.py
    import functools # I am using Python 2.5
    def identity_dec(func):
        def wrapper(*args, **kw):
            return func(*args, **kw)
        return functools.update_wrapper(wrapper, func)
    
    @identity_dec
    def example(): 
        pass
    >>> import inspect
    >>> from example import example
    >>> print inspect.getsource(example)
        def wrapper(*args, **kw):
            return func(*args, **kw)

    You get the source code of the closure
    and not what would be more meaningful, i.e.
    the string

    """
    @identity_dec
    def example():
    pass
    """

    Of course one could argue that this is not a bug
    (in a sense the inspect module is doing the right
    thing) but still it is giving information which
    is not very useful.

    Looking at the guts of inspect.getsource, one discovers the origin
    of the problem: inspect.findsource is looking at the attribute
    .co_firstlineno of the decorated function code object.
    Unfortunately .co_firstlineno is a read-only attribute, otherwise
    it would be possibile to change functools.update_wrapper to set it to
    the correct line number (i.e. the line where
    the undecorated function is defined, -1). So
    I don't think you can fix this in current
    Python, but it is something to keep in mind for Python 2.6 and 3.0. It should also manage classmethods/
    staticmethods and other decorators not implemented
    as closures.

    @micheles micheles mannequin added stdlib Python modules in the Lib dir labels Jul 31, 2007
    @tiran
    Copy link
    Member

    tiran commented Nov 23, 2007

    I'm setting the target to 2.6 and 3.0. Maybe somebody can come up with a
    sensible patch.

    @tiran tiran changed the title inspect.getsource does not work with decorated functions inspect.getsource does not work with decorated functions Nov 23, 2007
    @tiran tiran changed the title inspect.getsource does not work with decorated functions inspect.getsource does not work with decorated functions Nov 23, 2007
    @pitrou
    Copy link
    Member

    pitrou commented Jan 17, 2008

    Rather than devising something specific to the co_firstlineno attribute,
    why not have functools.update_wrapper add a "wrapped_func" attribute
    pointing to the original function object? That way, each function
    inspecting the decorated function would have the opportunity to walk the
    decoration chain if it wants.

    @gpolo
    Copy link
    Mannequin

    gpolo mannequin commented Feb 7, 2008

    I am attaching a patch that address this issue.

    @abalkin
    Copy link
    Member

    abalkin commented Jul 19, 2010

    I did not test the patch, but if it really addresses the issue, I don't see why such a simple fix would not be applied.

    Guilherme,

    Can you add unit tests? Also, the second chunk in your patch is just a style fix which is a distraction for reviewers.

    @abalkin abalkin self-assigned this Jul 19, 2010
    @abalkin abalkin added the type-bug An unexpected behavior, bug, or error label Jul 19, 2010
    @abalkin abalkin self-assigned this Jul 19, 2010
    @abalkin abalkin added the type-bug An unexpected behavior, bug, or error label Jul 19, 2010
    @ysjray
    Copy link
    Mannequin

    ysjray mannequin commented Jul 19, 2010

    I don't think this patch is correct, because we don't know if the closure[0] is the wrapped function, consider the following case:

    def fun():
        abc = 1
        def fun2():
            print(abc)
        print(inspect.getsource(fun2))

    In this case, the __closure__ of fun2 is [cell(1), ], the patch doesn't work.

    I think the behavior of inspect.getsource(arg) is no problem because it indeed gives the right source code of arg, no matter arg is a wrapper function or an original function. michele argues that the result is not meaningful, through in most case it seems reasonably because wrapper functions usually do litter valuable work, but it is not correct for all cases. What if a wrapper function does more valuable work than the wrapped function? In this case should getsouce() give the source code of wrapper function because it's more meaningful? The concept "wrapper" and "wrapped" should have no relation with its source code.

    I suggest you assign a special named attribute of all wrapper functions to its wrapped function yourself, something like "wrapped_function", and then you can walk through the wrapper chain to find the real source you want for each wrapper function. Then the stdard library's update_wrapper() and getsource() can remain unchanged.

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Sep 29, 2013

    Hello. Attached patch which uses the new inspect.unwrap, added in http://hg.python.org/cpython/rev/2aa6c1e35b8a.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Sep 26, 2014

    New changeset ad9cc6124a19 by Yury Selivanov in branch 'default':
    inspect: Fix getsource() to support decorated functions.
    https://hg.python.org/cpython/rev/ad9cc6124a19

    @1st1
    Copy link
    Member

    1st1 commented Sep 26, 2014

    Thanks for the bug report and patch! Committed to 3.5.

    @1st1 1st1 closed this as completed Sep 26, 2014
    @1st1 1st1 closed this as completed Sep 26, 2014
    @eric-wieser
    Copy link
    Mannequin

    eric-wieser mannequin commented Jul 31, 2018

    This now leaves inspect.getsource inconsistent with inspect.getsourcefile:

    >> import inspect
    >> from contextlib import contextmanager

    >>> @contextmanager
    ... def func():
    ...    yield
    
    >>> inspect.getsource(func)
    '@contextmanager\ndef func():\n    yield\n'
    >>>inspect.getsourcefile(func)
    'C:\\Program Files\\Python 3.5\\lib\\contextlib.py'

    Should getsourcefile be changed to match?

    This is causing numpy/numpy#11639, but it's not clear if this is a bug or by design.

    @1st1
    Copy link
    Member

    1st1 commented Jul 31, 2018

    Should getsourcefile be changed to match?

    I'd say yes. There's no point in getsourcefile returning the file location of the topmost decorator. Feel free to open a new issue and submit a PR to fix this!

    @eric-wieser
    Copy link
    Mannequin

    eric-wieser mannequin commented Aug 2, 2018

    New issue opened at https://bugs.python.org/issue34305, along with a PR linked there.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants