Title: Function source inspection fails on closures
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.6, Python 3.5
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: meador.inge Nosy List: bkcsfi sfi, malthe, meador.inge, ncoghlan, pitrou, python-dev, terry.reedy, yselivanov
Priority: normal Keywords: patch

Created on 2015-06-22 09:41 by malthe, last changed 2015-07-24 04:01 by meador.inge. This issue is now closed.

File name Uploaded Description Edit malthe, 2015-06-22 09:41 Python script that demonstrates bug
issue24485-v0.patch meador.inge, 2015-06-28 19:15 review
Messages (12)
msg245621 - (view) Author: Malthe Borch (malthe) * Date: 2015-06-22 09:41
Very simple to reproduce (see attachment).
msg245634 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-06-22 15:11
Looks like this is a regression in 3.5.
msg245862 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-06-26 19:43
I verified (Win7, but should be irrelevant) that works on 2.7.10 and 3.4.3.  The failure in 3.5.0b2 is omitting the body of inner.  Add another line to inner and both are omitted, so 'body' seems correct  Add another line to outer, and nothing is omitted.

def outer():
    def inner():

is displayed completely.  So the omission is the body of an inner function that is the last statement of outer.  This rule is not recursive, in the sense that for

def outer():
    def middle():
        def inner():

"def inner ..." is entirely omitted, not just 'inner1'.  The omission seems specific to 'def' as it does not occur with the other compound statements I tested (if, while).
msg245867 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-06-26 20:46
The test suite must lack, and therefore needs, a simple testcase as in

In 3.4 and 3.5, getsource() joins the list of lines returned by getsourcelines().  In both versions, getsourcelines uses findsource(), which seems to be unchanged.  In 3.5, the output of findsource for code, function, and method objects is postprocessed by _line_number_helper(code_object).  The bug applies to methods also.

class C:
    def outer():
        def inner():
from inspect import getsource

omits 'inner1', but getsource(C) does not.  I believe the regression is due to ac86e5b2d45b in #21217.  (At first glance, it would seem that the fixup in _line_number_helper should be in the code object part of findsource itself.)  I requested on that issue that the authors take a look at this.
msg245873 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-06-26 22:23
Thanks, Terry. I posted some findings to #21217.
msg245914 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2015-06-28 19:15
Here is a first cut patch that moves things back to using a syntax-level
analysis instead of a bytecode-level one.  I extended `BlockFinder`
to skip decorators.
msg247041 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-07-21 15:56
Meador, the patch looks OK.  Could you please commit it yourself?
msg247060 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2015-07-21 20:15
Will do.  Thanks for the review.
msg247203 - (view) Author: Roundup Robot (python-dev) Date: 2015-07-23 14:10
New changeset 4e42a62d5648 by Yury Selivanov in branch '3.5':
Issue #24485: Revert backwards compatibility breaking changes of #21217.

New changeset 98a2bbf2cce2 by Yury Selivanov in branch 'default':
Merge 3.5 (issues #21217, #24485).
msg247204 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-07-23 14:13
Meador, I've reverted changes introduced in #21217 -- I don't want to risk shipping 3.5beta4 with broken backwards compatibility.

Feel free to rebase & commit your patch (I decorated test_decorator_with_lambda with @unittest.expectedFailure).
msg247244 - (view) Author: Roundup Robot (python-dev) Date: 2015-07-24 04:00
New changeset 5400e21e92a7 by Meador Inge in branch '3.5':
Issue #24485: Function source inspection fails on closures.

New changeset 0e7d64595223 by Meador Inge in branch 'default':
Issue #24485: Function source inspection fails on closures.
msg247246 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2015-07-24 04:01
Thanks Yury!  I have committed my patches to 3.5 and default.
Date User Action Args
2015-07-24 04:01:16meador.ingesetstatus: open -> closed
resolution: fixed
messages: + msg247246

stage: patch review -> resolved
2015-07-24 04:00:15python-devsetmessages: + msg247244
2015-07-23 14:13:43yselivanovsetmessages: + msg247204
2015-07-23 14:10:52python-devsetnosy: + python-dev
messages: + msg247203
2015-07-21 20:15:48meador.ingesetmessages: + msg247060
2015-07-21 15:56:37yselivanovsetassignee: meador.inge
messages: + msg247041
2015-07-05 01:14:06larrysetpriority: release blocker -> normal
2015-06-28 19:15:44meador.ingesetfiles: + issue24485-v0.patch

nosy: + meador.inge
messages: + msg245914

keywords: + patch
stage: test needed -> patch review
2015-06-26 22:23:17yselivanovsetnosy: + ncoghlan, pitrou
messages: + msg245873
2015-06-26 20:46:38terry.reedysetmessages: + msg245867
stage: needs patch -> test needed
2015-06-26 19:43:43terry.reedysetnosy: + terry.reedy
messages: + msg245862
2015-06-22 15:11:05yselivanovsetpriority: normal -> release blocker

messages: + msg245634
stage: needs patch
2015-06-22 15:08:42yselivanovsetnosy: + yselivanov
2015-06-22 14:40:43bkcsfi sfisetnosy: + bkcsfi sfi
2015-06-22 09:41:28malthecreate