Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ast.Call end_lineno is defined and returns None #84903

Closed
gaborbernat mannequin opened this issue May 22, 2020 · 13 comments
Closed

ast.Call end_lineno is defined and returns None #84903

gaborbernat mannequin opened this issue May 22, 2020 · 13 comments
Labels
3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@gaborbernat
Copy link
Mannequin

gaborbernat mannequin commented May 22, 2020

BPO 40726
Nosy @serhiy-storchaka, @asottile, @pablogsal, @miss-islington, @remilapeyre, @gaborbernat, @isidentical, @laloch
PRs
  • bpo-40726: handle uninitalized end_lineno on ast.increment_lineno #20312
  • [3.9] bpo-40726: handle uninitalized end_lineno on ast.increment_lineno (GH-20312) #21741
  • [3.8] bpo-40726: handle uninitalized end_lineno on ast.increment_line… #21742
  • [3.8] bpo-40726: handle uninitalized end_lineno on ast.increment_lineno #21745
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-08-05.19:34:48.912>
    created_at = <Date 2020-05-22.08:34:59.646>
    labels = ['type-bug', 'library', '3.9', '3.10']
    title = 'ast.Call end_lineno is defined and returns None'
    updated_at = <Date 2020-08-05.19:34:48.911>
    user = 'https://github.com/gaborbernat'

    bugs.python.org fields:

    activity = <Date 2020-08-05.19:34:48.911>
    actor = 'pablogsal'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-08-05.19:34:48.912>
    closer = 'pablogsal'
    components = ['Library (Lib)']
    creation = <Date 2020-05-22.08:34:59.646>
    creator = 'gaborjbernat'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 40726
    keywords = ['patch']
    message_count = 13.0
    messages = ['369567', '369569', '369572', '369574', '369577', '369581', '369614', '369619', '369621', '369623', '374880', '374882', '374893']
    nosy_count = 8.0
    nosy_names = ['serhiy.storchaka', 'Anthony Sottile', 'pablogsal', 'miss-islington', 'remi.lapeyre', 'gaborjbernat', 'BTaskaya', 'laloch']
    pr_nums = ['20312', '21741', '21742', '21745']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue40726'
    versions = ['Python 3.9', 'Python 3.10']

    @gaborbernat
    Copy link
    Mannequin Author

    gaborbernat mannequin commented May 22, 2020

    Reporting an issue from xonsh/xonsh#3581; boils down to, ast.Call used to not define end_lineno in 3.8:

    py -3.8 -c 'import ast; type(ast.Call().end_lineno)'                                                                                                                                                                 Traceback (most recent call last):
      File "<string>", line 1, in <module>
    AttributeError: 'Call' object has no attribute 'end_lineno'

    However in 3.9 is defined with None:

    py -3.9 -c 'import ast; type(ast.Call().end_lineno)'

    This messes with some other operations in ast, namely https://github.com/python/cpython/blob/master/Lib/ast.py#L233

    @gaborbernat gaborbernat mannequin added 3.9 only security fixes labels May 22, 2020
    @laloch
    Copy link
    Mannequin

    laloch mannequin commented May 22, 2020

    The issue is not limited to ast.Call. Other AST nodes are also affected (e.g. ast.BinOp). It's also not limited to end_lineno attribute. The same applies to end_col_offset.

    @isidentical
    Copy link
    Sponsor Member

    This is because the 'end_lineno' and 'end_col_offset' are declared as optional attributes in the ASDL spec. The commit 'b7e9525f9c7ef02a1d2ad8253afdeb733b0951d4' made all optional fields and attributes auto initalized with None.

    @isidentical
    Copy link
    Sponsor Member

    See here for the complete spec: https://docs.python.org/3.9/library/ast.html#abstract-grammar

    @remilapeyre
    Copy link
    Mannequin

    remilapeyre mannequin commented May 22, 2020

    This was done in b7e9525 for bpo-36287, all attribute should now be defined, even when they are not set. It looks like some parts of the ast module where not updated.

    If nobody works on this I will send a PR to update the rest of AST later today.

    @remilapeyre remilapeyre mannequin added stdlib Python modules in the Lib dir 3.10 only security fixes type-bug An unexpected behavior, bug, or error labels May 22, 2020
    @isidentical
    Copy link
    Sponsor Member

    If nobody works on this I will send a PR to update the rest of AST later today.

    :/ I'm terribly sorry for that, I just sent a PR without refreshing the tab.

    @asottile
    Copy link
    Mannequin

    asottile mannequin commented May 22, 2020

    There's current expectation in a lot of linters / code formatters that the *lineno and *col_offset attributes will be missing if they are not attached to the node

    setting them to None is going to break a lot of those, if possible I'd suggest going back to when they were missing

    @isidentical
    Copy link
    Sponsor Member

    setting them to None is going to break a lot of those, if possible I'd suggest going back to when they were missing

    'lineno' and 'col_offset' will never be none, since both are declared as normal integers. But on the other hand, 'end_lineno' and 'end_col_offset' are declared as optional integers which would make sense to auto initalize them with *None*, and I dont think it would ever break some code that complies with the ASDL declaration.

    @laloch
    Copy link
    Mannequin

    laloch mannequin commented May 22, 2020

    Actually, Xonsh (http://github.com/xonsh/xonsh) tests show that keyword AST nodes are missing 'lineno' attribute, but that could be our fault.

    @laloch
    Copy link
    Mannequin

    laloch mannequin commented May 22, 2020

    Actually, Xonsh tests show that keyword AST nodes are missing 'lineno' attribute, but that could be our fault.

    Yes, our fault. Sorry for the noise.

    @pablogsal
    Copy link
    Member

    New changeset 8f4380d by Batuhan Taskaya in branch 'master':
    bpo-40726: handle uninitalized end_lineno on ast.increment_lineno (GH-20312)
    8f4380d

    @miss-islington
    Copy link
    Contributor

    New changeset a132098 by Miss Islington (bot) in branch '3.9':
    bpo-40726: handle uninitalized end_lineno on ast.increment_lineno (GH-20312)
    a132098

    @miss-islington
    Copy link
    Contributor

    New changeset b24c9d2 by Batuhan Taskaya in branch '3.8':
    [3.8] bpo-40726: handle uninitalized end_lineno on ast.increment_lineno (GH-21745)
    b24c9d2

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants