Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add line and column information for keywords in the AST #84322

Closed
pablogsal opened this issue Apr 1, 2020 · 4 comments
Closed

Add line and column information for keywords in the AST #84322

pablogsal opened this issue Apr 1, 2020 · 4 comments
Labels
3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@pablogsal
Copy link
Member

BPO 40141
Nosy @pablogsal, @isidentical
PRs
  • bpo-40141: Add line and column information to keyword AST nodes #19283
  • bpo-40141: Include the value in column position for the keyword AST nodes #19348
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-04-02.00:06:26.434>
created_at = <Date 2020-04-01.20:49:16.839>
labels = ['interpreter-core', '3.9']
title = 'Add line and column information for keywords in the AST'
updated_at = <Date 2020-04-03.20:02:33.510>
user = 'https://github.com/pablogsal'

    bugs.python.org fields:

    activity = <Date 2020-04-03.20:02:33.510>
actor = 'pablogsal'
assignee = 'none'
closed = True
closed_date = <Date 2020-04-02.00:06:26.434>
closer = 'pablogsal'
components = ['Interpreter Core']
creation = <Date 2020-04-01.20:49:16.839>
creator = 'pablogsal'
dependencies = []
files = []
hgrepos = []
issue_num = 40141
keywords = ['patch']
message_count = 4.0
messages = ['365509', '365510', '365531', '365720']
nosy_count = 2.0
nosy_names = ['pablogsal', 'BTaskaya']
pr_nums = ['19283', '19348']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40141'
versions = ['Python 3.9']
    @pablogsal
    Copy link
    Member Author

    When inspecting keyword parameters in a function call, the keyword is stored as a string and not as a AST node:

    >>> import ast
>>> r = "f(a,   xxxxxxxxxxa  =  34, y=23)"
>>> node = ast.parse(r)
>>> ll = node.body[0].value.keywords[0].arg
>>> node.body[0].value.keywords[0].arg
'xxxxxxxxxxa'

    this makes impossible to locate the keyword in the code using the AST as it lacks AST attributes (lineno, col_offset, end_lineno, end_col_offset).

    On the other hand, this does not happen with args, that has the meta-information:

    >>> node.body[0].value.args[0].id
'a'
>>> dir(node.body[0].value.args[0])

    So one can locate the arg string using:

    >>> r[arg.col_offset:arg.end_col_offset]
'a'

    For this reason, we should add the same information to keyword nodes.

    @pablogsal pablogsal added 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Apr 1, 2020
    @pablogsal
    Copy link
    Member Author

    I am preparing more PRs for other nodes that are missing the meta-information as well but will open them in a separate issue.

    @pablogsal
    Copy link
    Member Author

    New changeset 168660b by Pablo Galindo in branch 'master':
    bpo-40141: Add line and column information to ast.keyword nodes (GH-19283)
    168660b

    @pablogsal
    Copy link
    Member Author

    New changeset 40cf35c by Pablo Galindo in branch 'master':
    bpo-40141: Include the value in the column position for keyword AST nodes (GH-19348)
    40cf35c

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @pablogsal