Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid mapping patterns give confusing SyntaxErrors #88534

Closed
brandtbucher opened this issue Jun 9, 2021 · 21 comments
Closed

Invalid mapping patterns give confusing SyntaxErrors #88534

brandtbucher opened this issue Jun 9, 2021 · 21 comments
Labels
3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@brandtbucher
Copy link
Member

BPO 44368
Nosy @lysnikolaou, @pablogsal, @miss-islington, @brandtbucher
PRs
  • bpo-44368: Ensure we don't raise incorrect custom syntax errors with soft keywords #26630
  • [3.10] bpo-44368: Ensure we don't raise incorrect custom syntax errors with soft keywords (GH-26630) #26631
  • bpo-44368: Improve syntax errors with invalid as pattern targets #26632
  • [3.10] bpo-44368: Improve syntax errors with invalid as pattern targets (GH-26632) #26792
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2021-06-09.19:47:35.477>
    labels = ['interpreter-core', 'type-bug', '3.11']
    title = 'Invalid mapping patterns give confusing SyntaxErrors'
    updated_at = <Date 2021-06-18.21:16:02.181>
    user = 'https://github.com/brandtbucher'

    bugs.python.org fields:

    activity = <Date 2021-06-18.21:16:02.181>
    actor = 'pablogsal'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Parser']
    creation = <Date 2021-06-09.19:47:35.477>
    creator = 'brandtbucher'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 44368
    keywords = ['patch']
    message_count = 20.0
    messages = ['395453', '395454', '395458', '395460', '395461', '395462', '395467', '395469', '395470', '395473', '395477', '395478', '395479', '395481', '395482', '395485', '395486', '395487', '395584', '396092']
    nosy_count = 4.0
    nosy_names = ['lys.nikolaou', 'pablogsal', 'miss-islington', 'brandtbucher']
    pr_nums = ['26630', '26631', '26632', '26792']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue44368'
    versions = ['Python 3.11']

    @brandtbucher
    Copy link
    Member Author

    Here are a few that I found. Not sure when they were introduced:

    match ...:
    case {**rest, "key": value}:
    pass

    match ...:
    case {"first": first, **rest, "last": last}:
    pass

    match ...:
    case {**_}:
    pass

    These all give the following error while parsing the second line:

    File "<stdin>", line 1
    match ...:
    ^^^^^^^^^
    SyntaxError: invalid syntax. Perhaps you forgot a comma?

    @brandtbucher brandtbucher added 3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Jun 9, 2021
    @brandtbucher
    Copy link
    Member Author

    Perhaps we need something like invalid_mapping_pattern or invalid_mapping_pattern_double_star rules?

    @pablogsal
    Copy link
    Member

    Probably, otherwise is going to hurt because the syntax errors that you describe trigger when two names are place together, which in general is a missing comma.

    The error message also doesn't say: "you are missing a comma" it says that the most typical reason is a missing comma, which is important to distinguish :)

    @pablogsal
    Copy link
    Member

    The backtracking with the soft keyword may make this very annoying, by the way.

    @pablogsal
    Copy link
    Member

    Oh, wait, I think I misunderstood the problem. The problem is that the parser is backtracking and identifying match as a name.

    Indeed, the soft keyword is a pain :(

    @pablogsal
    Copy link
    Member

    I think it actually will be very useful to explain that these cases are invalid in the Syntax error, which will also solve this problem.

    This on the other hand shows a bigger problem: any generic syntax error that happens inside "match" will probably end identifying the keyword as a name, even if we don't have specific errors, the parser will point to it and complain about "match".

    @pablogsal
    Copy link
    Member

    Oh, turns out I already added machinery to solved this but I was missing a piece!

    @brandtbucher
    Copy link
    Member Author

    Wow, that was quite a roller-coaster ride. Thanks Pablo. :)

    @brandtbucher
    Copy link
    Member Author

    brandtbucher commented Jun 9, 2021

    I found a similar one, by the way (not related to mapping patterns):

    >>> match ...:
    ...     case 42 as _:
    ...         pass
    
      File "<stdin>", line 2
        case 42 as _:
                   ^
    SyntaxError: expected ':'

    Is this covered by your fix?

    @pablogsal
    Copy link
    Member

    Is this covered by your fix?

    No, that is not a backtracking error (is not going all the way to the "match" expression).

    @pablogsal
    Copy link
    Member

    I will fix that one in another PR

    @pablogsal
    Copy link
    Member

    New changeset 457ce60 by Pablo Galindo in branch 'main':
    bpo-44368: Ensure we don't raise incorrect custom syntax errors with soft keywords (GH-26630)
    457ce60

    @pablogsal
    Copy link
    Member

    Oh, this one is actually correct:

    match ...:
    case 42 as _:
    pass

    File "<stdin>", line 2
    case 42 as _:
    ^
    SyntaxError: expected ':'

    That is literally expecting a ":" and that's the error. It has identified "case 42" correctly and it now expects a ":". Is just that the error marker is wrong because it has reached the "as and _" as part of the parsing so the error is there.

    That is going to be a bit tricky to "fix"

    @brandtbucher
    Copy link
    Member Author

    Could we just try parsing "as _" and raise if so? That wouldn't conflict with any existing rules, and that way we could actually have a helpful error message.

    @brandtbucher
    Copy link
    Member Author

    Like "SyntaxError: can't capture into a wildcard (consider removing)".

    @miss-islington
    Copy link
    Contributor

    New changeset f807a4f by Miss Islington (bot) in branch '3.10':
    bpo-44368: Ensure we don't raise incorrect custom syntax errors with soft keywords (GH-26630)
    f807a4f

    @pablogsal
    Copy link
    Member

    Could we just try parsing "as _" and raise if so? That wouldn't conflict with any existing rules, and that way we could actually have a helpful error message.

    No, the same happens with other targets such as:

    File "/home/pablogsal/github/python/master/lel.py", line 2
    case 42 as 1+1:
    ^
    SyntaxError: expected ':'

    @pablogsal
    Copy link
    Member

    Checkout PR 26632, and see if this works for you

    @pablogsal
    Copy link
    Member

    New changeset 0507303 by Pablo Galindo in branch 'main':
    bpo-44368: Improve syntax errors with invalid as pattern targets (GH-26632)
    0507303

    @pablogsal
    Copy link
    Member

    New changeset a8c418d by Pablo Galindo in branch '3.10':
    [3.10] bpo-44368: Improve syntax errors with invalid as pattern targets (GH-26632) (GH-26792)
    a8c418d

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @furkanonder
    Copy link
    Sponsor Contributor

    @pablogsal @brandtbucher The issue seems to be solved. We can close the issue.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants