This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Line number of docstring in AST
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: methane, myint, serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2017-05-28 18:44 by myint, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (5)
msg294654 - (view) Author: Steven Myint (myint) * Date: 2017-05-28 18:44
Since #29463, it is no longer obvious how to get the line number of a docstring in the AST:

    import ast

    x = ast.parse('''\
    def foo():
        """This is a docstring."""
    ''')

    # In Python 3.6, the docstring and line number would be:
    print(x.body[0].body[0].value.s)
    print(x.body[0].body[0].value.lineno)

    # In Python 3.7, I don't know what the equivalent would be.
    print(x.body[0].docstring)
    # Line number?

We use this feature in pyflakes (https://github.com/PyCQA/pyflakes/issues/271).
msg294673 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-05-29 06:33
It is not possible anymore. Why you need the line number of a docstring?

The closest approximation is between node.lineno and node.body[0].lineno (if node.body is not empty).
msg294688 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-29 10:32
> We use this feature in pyflakes (https://github.com/PyCQA/pyflakes/issues/271)

AST is not designed to be 1-to-1 .py source to AST mapping. If you need the exact line number, you might use your own parser. I don't know what is using pylint for example? There is also https://github.com/PyCQA/redbaron which provides such 1-to-1 mapping but it has a different API than AST.

Sadly, I suggest to close this issue as WONTFIX.
msg294706 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-05-29 17:41
Even if the line number of a docstring is known, it is not easy to determine the line number corresponding to the particular line in a docstring if it contains backslashes following by newline.

'''foo
bar
'''

and

'''\
foo
bar
'''

are equal strings, but the lines 'bar' have different offsets from the start of the strings.

The only robust method was parsing the source code starting from the start of a docstring. Now you just need to start parsing from the start of the function/class.
msg294708 - (view) Author: Steven Myint (myint) * Date: 2017-05-29 17:59
I think what you guys have brought up makes sense.

Now that you mention it, I see that pyflakes gives the wrong line number if an escaped newline appears after the doctests. Though, it works fine if the escaped newline appears before it.

https://github.com/PyCQA/pyflakes/blob/1af4f14ad4675bf5c61c47bbb7c2421b50d1cba4/pyflakes/checker.py#L842

This is a non-default feature in pyflakes anyway, so we can live with it.

Thanks
History
Date User Action Args
2022-04-11 14:58:47adminsetgithub: 74682
2017-05-29 17:59:27myintsetmessages: + msg294708
2017-05-29 17:41:19serhiy.storchakasetstatus: open -> closed
resolution: wont fix
messages: + msg294706

stage: resolved
2017-05-29 10:32:50vstinnersetmessages: + msg294688
2017-05-29 06:33:14serhiy.storchakasetnosy: + vstinner, methane, serhiy.storchaka
messages: + msg294673
2017-05-28 18:44:26myintcreate