This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: compile() can compile a bare starred expression with `PyCF_ONLY_AST` flag with the old parser, but not the new one
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.10, Python 3.9
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: BTaskaya, adamwill, benjamin.peterson, brett.cannon, christian.heimes, pablogsal, serhiy.storchaka, yselivanov
Priority: normal Keywords:

Created on 2020-06-02 19:08 by adamwill, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (4)
msg370620 - (view) Author: Adam Williamson (adamwill) Date: 2020-06-02 19:08
Not 100% sure this would be considered a bug, but it seems at least worth filing to check. This is a behaviour difference between the new parser and the old one. It's very easy to reproduce:

<mock-chroot> sh-5.0# PYTHONOLDPARSER=1 python3
Python 3.9.0b1 (default, May 29 2020, 00:00:00) 
[GCC 10.1.1 20200507 (Red Hat 10.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from _ast import *
>>> compile("(*starred)", "<unknown>", "exec", flags=PyCF_ONLY_AST)
<ast.Module object at 0x7fe1504532e0>
>>> 
<mock-chroot> sh-5.0# python3
Python 3.9.0b1 (default, May 29 2020, 00:00:00) 
[GCC 10.1.1 20200507 (Red Hat 10.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from _ast import *
>>> compile("(*starred)", "<unknown>", "exec", flags=PyCF_ONLY_AST)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<unknown>", line 1
    (*starred)
              ^
SyntaxError: invalid syntax

That is, you can compile() the expression "(*starred)" with PyCF_ONLY_AST flag set with the old parser, but not with the new one. Without PyCF_ONLY_AST you get a SyntaxError with both parsers, though a with the old parser, the error message is "can't use starred expression here", not "invalid syntax".
msg370625 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-06-02 20:16
There is no promise that the parser can parse invalid Python code. I think there is no a bug here.
msg370626 - (view) Author: Adam Williamson (adamwill) Date: 2020-06-02 20:28
Realized I forgot to give it, so in case it's important, the context here is the black test suite:

https://github.com/psf/black/issues/1441

that test suite has a file full of expressions that it expects to be able to parse this way (it uses `ast.parse()`, which in turn calls `compile()` with this flag). A bare (*starred) line is part of that file:

https://github.com/psf/black/blob/master/tests/data/expression.py#L149

and has been for as long as black has existed. Presumably if this isn't going to be fixed we'll need to adapt this black test file to test a starred expression in a 'valid' way, somehow.
msg370643 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-06-03 06:22
Yeah, this is not a bug as Serhiy points out. Regarding the error message, we are currently working on trying to unify the error messages in the the parser as much as we can but there is also no promise that they will match 100%.

Many grammar checks currently are done outside the parser (we mention it on  PEP617) so I am not surprised that the old parser is ok with some invalid expressions and PyCF_ONLY_AST. This is one of the things we are trying to change with the new parser precisely.

I will close for now as "not a bug". Feel free to reopen if you think we missed something or if something was left to discuss.
History
Date User Action Args
2022-04-11 14:59:31adminsetgithub: 85025
2020-06-03 06:22:40pablogsalsetstatus: open -> closed
resolution: not a bug
messages: + msg370643

stage: resolved
2020-06-02 20:28:11adamwillsetmessages: + msg370626
2020-06-02 20:16:34serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg370625
2020-06-02 19:59:26christian.heimessetnosy: + christian.heimes
2020-06-02 19:58:30christian.heimessetnosy: + brett.cannon, benjamin.peterson, yselivanov, pablogsal

type: behavior
versions: + Python 3.10
2020-06-02 19:08:35BTaskayasetnosy: + BTaskaya
2020-06-02 19:08:16adamwillcreate