classification
Title: help() function incorrectly captures comment preceding a nested function
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Gwenlliana, anupama.srinivas.murthy, iritkatriel, r.david.murray, raulcd, rhettinger
Priority: normal Keywords:

Created on 2015-01-10 09:33 by rhettinger, last changed 2021-11-30 00:15 by iritkatriel.

Messages (7)
msg233811 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2015-01-10 09:33
The help() function mysteriously captures a comment on the line preceding an inner function definition in a nested scope.

Given this code:
----------------

    def outer(func):
        #comment
        def inner():
            return
        return inner

    @outer
    def f():
        return


Calling help(f) produces:
-------------------------

    Help on function inner in module __main__:

    inner()
        #comment
msg234257 - (view) Author: anupama srinivas murthy (anupama.srinivas.murthy) * Date: 2015-01-18 16:31
The comment is captured only in the absence of a docstring within the function. The capture happens whether the function is nested or otherwise
msg234273 - (view) Author: Gwenlliana (Gwenlliana) Date: 2015-01-18 20:22
The capture actually worked correctly. It seems to be caused by the artifacts introduced by the evaluation function decorators.

Decorator lists for functions are compiled to a series of high-order function applications to the original function, followed by an assignment which stores the resulting function object to the name of the original function. A decompilation of the sample code might be something like this:

  3           0 LOAD_CONST               0 (<code object outer at ...>)
              3 MAKE_FUNCTION            0
              6 STORE_NAME               0 (outer)

 10           9 LOAD_NAME                0 (outer)
             12 LOAD_CONST               1 (<code object f ...>)
             15 MAKE_FUNCTION            0
             18 CALL_FUNCTION            1
             21 STORE_NAME               1 (f)
             24 LOAD_CONST               2 (None)
             27 RETURN_VALUE        

It works in the same way with `f = outer(lambda: None)`. Thus evaluating `f.__name__` or `f.func_name` would actually gives the __name__ property of that inner function object, thus `help(f)` actually queries the help information of function inner.

Similarly, lots of properties of the original function object f would be shadowed by the decoration process, including the __doc__ property. You'll find documenting f takes no effect on the value of `f.__doc__`:

    @outer
    def f():
    """ The docstring of function f """
        return

>>> help(f)
Help on function inner in module tmp:

inner()
    #comment
msg234275 - (view) Author: Gwenlliana (Gwenlliana) Date: 2015-01-18 20:41
Though standard library contains a workaround on the decorator issue (functools.wraps), but it still failed to patch func_code.co_firstlineno, which led the pydoc module to capture the wrong comment.
msg234311 - (view) Author: anupama srinivas murthy (anupama.srinivas.murthy) * Date: 2015-01-19 10:16
In Python 2.7, the capture happens even if there is no decorator. The code:
#Hey this is f
def f():

 return
 
help(f)
gives the output:
Help on function f in module __main__:

f()
    #Hey this is f
whereas a docstring inside the function causes the comment to be ignored. Therefore, the code:
#Hey this is f
def f():
 '''this is the function f'''
 return
 
help(f)
gives the output:
Help on function f in module __main__:

f()
    this is the function f
@Gwenllina:I need to clarify my previous comment. The docstring that would cause the preceding comment to be ignored must be in the inner function in case of the first example. Placing it in f() still causes comment capture as you said
msg241151 - (view) Author: Raúl Cumplido (raulcd) * Date: 2015-04-15 20:05
I am not sure what the expected behavior is. Based on the code on pydoc.py we can find:
def getdoc(object):
    """Get the doc string or comments for an object."""
    result = inspect.getdoc(object) or inspect.getcomments(object)

So if the doc string is not found it searches for the lines of comments immediately preceding the object's source code.

This is why the example from Anupama returns the preceding comment on both python2.7 and python3.5 (not only on python2.7).

@rhettinger this seems like a deliberate decision to add as help documentation the preceding comment if the docstring is not found. What were you expecting? Should this be changed?
msg407341 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-11-30 00:15
Reproduced on 3.11.
History
Date User Action Args
2021-11-30 00:15:29iritkatrielsetnosy: + iritkatriel

messages: + msg407341
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.2, Python 3.3, Python 3.5
2015-04-15 20:05:56raulcdsetnosy: + raulcd
messages: + msg241151
2015-01-19 10:16:56anupama.srinivas.murthysetmessages: + msg234311
2015-01-18 20:41:26Gwenllianasetmessages: + msg234275
2015-01-18 20:22:15Gwenllianasetnosy: + Gwenlliana
messages: + msg234273
2015-01-18 16:31:59anupama.srinivas.murthysetnosy: + anupama.srinivas.murthy
messages: + msg234257
2015-01-10 15:26:11r.david.murraysetnosy: + r.david.murray
2015-01-10 09:33:16rhettingercreate