This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Inconsistency between Python 2 and PyPy regarding future imports
Type: behavior Stage: resolved
Components: Versions: Python 2.7
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: BTaskaya, Claudiu.Popa, benjamin.peterson, brett.cannon, terry.reedy
Priority: normal Keywords:

Created on 2014-10-10 20:48 by Claudiu.Popa, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (4)
msg229037 - (view) Author: PCManticore (Claudiu.Popa) * (Python triager) Date: 2014-10-10 20:48

The following code gives similar errors for both PyPy and Python 2 when called, but totally different AST objects.

Given the following code, PyPy says that `print(4)` is an _ast.Print, while Python 2 believes that it is an _ast.Expr (which wraps a Call). This behaviour is wrong, Python 2 shouldn't extrapolate further if the __future__ is misplaced.

   from ast import parse

   bad = """
   a = 1
   from __future__ import print_function

   x = parse(bad)

Brett, I added you on this as nosy, since I discovered this bug while debugging your print-statement pull request, so I thought you should know. :-)
msg229626 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-10-18 01:28
A difference in internal behavior between CPython and PyPy is not necessarily a CPython bug, or even a PyPy bug.  It may just be an implementation choice.  So ignore PyPy and consider Python doc versus CPython behavior.

Running the three quoted lines by themselves gives SyntaxError.  compiling the ast object does the same.  So this is one of those SyntaxErrors not noticed (or at least not enforced) until the compile phase. This was specified in PEP 236 and still is in the Reference: "A future statement is recognized and treated specially at compile time".  Whenever the compiler raises SyntaxError, it declares the AST to be syntactically invalid.  How can this be?

The ast doc starts with "The ast module helps Python applications to process trees of the Python *abstract syntax grammar*. ... this module helps to find out programmatically what the current grammar looks like." (asterisks added). The abstract grammar does *not* include a 'Future' statement. In your code, x.body[1] is an ImportFrom object.  So the ast from the code above does not violate the abstract grammar and there is no parsing bug.

We should either either close this as 'not a bug' or change to a doc issue to add something about the ast abstract grammar being 'looser' than the defacto context-sensitive Python grammar.  Perhaps something like "The language defined by the context-free LL(1) abstract grammer is a superset of the actual context-sensitive Python language. (Consider position-dependent future statements.)  Consequently, some ast trees generate a SyntaxError when compiled."

It is somewhat arbitrary when a print statement is converted to a print expression -- while parsing, or while compiling.  The two implementations made different choices.

Since the parser does recognize a Future statement and change the ast objects it creates, it would be possible to add a Future object to ast trees.  But it would do no good as far as the syntax check goes.  The context-free LL(1) parser could not recognize a violation of the context-sensitive constraint.
msg359464 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2020-01-06 20:04
I guess this issue expired (due to python2).
msg359488 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2020-01-07 02:36
If you fully compiled this code, it would fail. CPython checks the well-formedness of __future__ statements after ast parsing.
Date User Action Args
2022-04-11 14:58:09adminsetgithub: 66796
2020-01-07 02:36:42benjamin.petersonsetstatus: open -> closed
resolution: not a bug
messages: + msg359488

stage: resolved
2020-01-06 20:04:13BTaskayasetnosy: + BTaskaya
messages: + msg359464
2014-10-18 01:28:33terry.reedysetnosy: + terry.reedy, benjamin.peterson
messages: + msg229626
2014-10-10 20:48:45Claudiu.Popacreate