This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ast.parse fails to trigger SyntaxWarning that normal execution does
Type: behavior Stage: resolved
Components: Versions: Python 3.9, Python 3.8
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Anthony Sottile, BTaskaya, serhiy.storchaka
Priority: normal Keywords:

Created on 2020-04-24 18:18 by Anthony Sottile, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (6)
msg367207 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2020-04-24 18:18
compare the following:

```
if False: 'foo'(1)
```


$ python3.9
Python 3.9.0a5 (default, Mar 23 2020, 23:11:30) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> ast.parse(b"if False: 'foo'(1)")
<ast.Module object at 0x7ff87a03cca0>
>>> if False: 'foo'(1)
... 
<stdin>:1: SyntaxWarning: 'str' object is not callable; perhaps you missed a comma?


even with `PYTHONWARNINGS=error` no warning / error is raised
msg367229 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2020-04-24 21:15
This is probably because that these warnings raised during code generation, rather then AST Validation / generation.
msg367231 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-04-24 21:25
It is not a bug. Execution includes more stages than ast.parse().
msg367232 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2020-04-24 21:41
I would really like to be able to trigger all warnings consistently and statically without executing the code.

This is useful (for instance) for this simple script which validates the ast: https://github.com/pre-commit/pre-commit-hooks/blob/master/pre_commit_hooks/check_ast.py

when run with PYTHONWARNINGS=error I get nice error reports for things such as invalid escape sequence, but I do not get them for string-call which I would like to

This does seem like a bug and prematurely closed, I'm a bit disappointed in the current resolution
msg367240 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2020-04-24 23:50
> This does seem like a bug and prematurely closed, I'm a bit disappointed in the current resolution

I would love to see it moved as a new step in the execution pipeline that we cane extend easily and use externally (like PyAST_Validate). But I'm not sure if the burden worths the gain. Though, for the current version it would be very simple to implement such checks (3-4) with a simple NodeVisitor. Just an example (should be pretty close to output, not exact): https://github.com/isidentical/astvalidate/blob/master/astvalidate/validators/syntatical.py
msg367260 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-04-25 06:50
Compilation includes the following stages:

1. Tokenizing.
2. Creating CST (concrete syntax tree).
3. Creating AST (abstract syntax tree).
4. Optimizing AST.
5. Building symtable.
6. Generating bytecode.
7. Optimizing bytecode.

ast.parse() only includes stages 1-3. Many warnings and errors can be raised at stages 5 and 6 (for example errors for "break" outside of a loop and "unlocal" at module level). If you want to get all warnings and errors, use compile().
History
Date User Action Args
2022-04-11 14:59:29adminsetgithub: 84558
2020-04-25 06:50:06serhiy.storchakasetmessages: + msg367260
2020-04-24 23:50:50BTaskayasetmessages: + msg367240
2020-04-24 21:41:41Anthony Sottilesetmessages: + msg367232
2020-04-24 21:25:08serhiy.storchakasetstatus: open -> closed

nosy: + serhiy.storchaka
messages: + msg367231

resolution: not a bug
stage: resolved
2020-04-24 21:15:46BTaskayasetnosy: + BTaskaya
messages: + msg367229
2020-04-24 18:18:04Anthony Sottilecreate