classification
Title: Invalid mapping patterns give confusing SyntaxErrors
Type: behavior Stage: patch review
Components: Parser Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: brandtbucher, lys.nikolaou, miss-islington, pablogsal
Priority: normal Keywords: patch

Created on 2021-06-09 19:47 by brandtbucher, last changed 2021-06-10 22:50 by pablogsal.

Pull Requests
URL Status Linked Edit
PR 26630 merged pablogsal, 2021-06-09 20:42
PR 26631 merged miss-islington, 2021-06-09 21:20
PR 26632 merged pablogsal, 2021-06-09 21:58
Messages (19)
msg395453 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-06-09 19:47
Here are a few that I found. Not sure when they were introduced:

match ...:
    case {**rest, "key": value}:
        pass

match ...:
    case {"first": first, **rest, "last": last}:
        pass

match ...:
    case {**_}:
        pass

These all give the following error while parsing the second line:

  File "<stdin>", line 1
    match ...:
    ^^^^^^^^^
SyntaxError: invalid syntax. Perhaps you forgot a comma?
msg395454 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-06-09 19:48
Perhaps we need something like invalid_mapping_pattern or invalid_mapping_pattern_double_star rules?
msg395458 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-09 20:17
Probably, otherwise is going to hurt because the syntax errors that you describe trigger when two names are place together, which in general is a missing comma.

The error message also doesn't say: "you are missing a comma" it says that the most typical reason is a missing comma, which is important to distinguish :)
msg395460 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-09 20:18
The backtracking with the soft keyword may make this very annoying, by the way.
msg395461 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-09 20:20
Oh, wait, I think I misunderstood the problem. The problem is that the parser is backtracking and identifying match as a name. 

Indeed, the soft keyword is a pain :(
msg395462 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-09 20:22
I think it actually will be very useful to explain that these cases are invalid in the Syntax error, which will also solve this problem.

This on the other hand shows a bigger problem: any generic syntax error that happens inside "match" will probably end identifying the keyword as a name, even if we don't have specific errors, the parser will point to it and complain about "match".
msg395467 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-09 20:42
Oh, turns out I already added machinery to solved this but I was missing a piece!
msg395469 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-06-09 21:05
Wow, that was quite a roller-coaster ride. Thanks Pablo. :)
msg395470 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-06-09 21:08
I found a similar one, by the way (not related to mapping patterns):

match ...:
    case 42 as _:
        pass

  File "<stdin>", line 2
    case 42 as _:
               ^
SyntaxError: expected ':'

Is this covered by your fix?
msg395473 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-09 21:12
> Is this covered by your fix?

No, that is not a backtracking error (is not going all the way to the "match" expression).
msg395477 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-09 21:16
I will fix that one in another PR
msg395478 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-09 21:20
New changeset 457ce60fc70f1c9290023f46fb82b6a490dff32e by Pablo Galindo in branch 'main':
bpo-44368: Ensure we don't raise incorrect custom syntax errors with soft keywords (GH-26630)
https://github.com/python/cpython/commit/457ce60fc70f1c9290023f46fb82b6a490dff32e
msg395479 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-09 21:24
Oh, this one is actually correct:

match ...:
    case 42 as _:
        pass

  File "<stdin>", line 2
    case 42 as _:
               ^
SyntaxError: expected ':'

That is literally expecting a ":" and that's the error. It has identified "case 42" correctly and it now expects a ":". Is just that the error marker is wrong because it has reached the "as and _" as part of the parsing so the error is there.

That is going to be a bit tricky to "fix"
msg395481 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-06-09 21:31
Could we just try parsing "as _" and raise if so? That wouldn't conflict with any existing rules, and that way we could actually have a helpful error message.
msg395482 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-06-09 21:32
Like "SyntaxError: can't capture into a wildcard (consider removing)".
msg395485 - (view) Author: miss-islington (miss-islington) Date: 2021-06-09 21:45
New changeset f807a4fad4da8f629ea7fe1f00719e817c77b63c by Miss Islington (bot) in branch '3.10':
bpo-44368: Ensure we don't raise incorrect custom syntax errors with soft keywords (GH-26630)
https://github.com/python/cpython/commit/f807a4fad4da8f629ea7fe1f00719e817c77b63c
msg395486 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-09 21:57
> Could we just try parsing "as _" and raise if so? That wouldn't conflict with any existing rules, and that way we could actually have a helpful error message.

No, the same happens with other targets such as:

 File "/home/pablogsal/github/python/master/lel.py", line 2
    case 42 as 1+1:
               ^
SyntaxError: expected ':'
msg395487 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-09 21:59
Checkout PR 26632, and see if this works for you
msg395584 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-10 22:50
New changeset 05073036dcecefc00b0c3e7397601809da41e2f1 by Pablo Galindo in branch 'main':
bpo-44368: Improve syntax errors with invalid as pattern targets (GH-26632)
https://github.com/python/cpython/commit/05073036dcecefc00b0c3e7397601809da41e2f1
History
Date User Action Args
2021-06-10 22:50:39pablogsalsetmessages: + msg395584
2021-06-09 21:59:14pablogsalsetmessages: + msg395487
2021-06-09 21:58:49pablogsalsetpull_requests: + pull_request25218
2021-06-09 21:57:52pablogsalsetmessages: + msg395486
2021-06-09 21:45:56miss-islingtonsetnosy: + miss-islington
messages: + msg395485
2021-06-09 21:32:12brandtbuchersetmessages: + msg395482
2021-06-09 21:31:14brandtbuchersetmessages: + msg395481
2021-06-09 21:24:52pablogsalsetnosy: - miss-islington
messages: + msg395479
2021-06-09 21:20:11miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25217
2021-06-09 21:20:11pablogsalsetmessages: + msg395478
2021-06-09 21:16:52pablogsalsetmessages: + msg395477
2021-06-09 21:12:16pablogsalsetmessages: + msg395473
2021-06-09 21:08:51brandtbuchersetmessages: + msg395470
2021-06-09 21:05:37brandtbuchersetmessages: + msg395469
2021-06-09 20:42:55pablogsalsetkeywords: + patch
stage: patch review
pull_requests: + pull_request25216
2021-06-09 20:42:26pablogsalsetmessages: + msg395467
2021-06-09 20:22:25pablogsalsetmessages: + msg395462
2021-06-09 20:20:04pablogsalsetmessages: + msg395461
2021-06-09 20:18:37pablogsalsetmessages: + msg395460
2021-06-09 20:17:31pablogsalsetmessages: + msg395458
2021-06-09 19:48:43brandtbuchersetmessages: + msg395454
2021-06-09 19:47:35brandtbuchercreate