Title: __future__ imports fail when compiling from python ast
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.2, Python 2.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, brett.cannon, georg.brandl, ncoghlan, python-dev, r.david.murray, talljosh
Priority: release blocker Keywords:

Created on 2012-03-21 06:19 by talljosh, last changed 2012-03-22 12:58 by benjamin.peterson. This issue is now closed.

File name Uploaded Description Edit talljosh, 2012-03-21 06:19 Python code demonstrating the issue
Messages (6)
msg156477 - (view) Author: J. D. Bartlett (talljosh) Date: 2012-03-21 06:19
I am trying to compile an AST which contains an ImportFrom node which performs a __future__ import. The ImportFrom node in question is the first node within the AST's body (as it should be, because __future__ imports must occur at the beginning of modules).

Compiling the AST fails with the error "SyntaxError: from __future__ imports must occur at the beginning of the file".

The future_parse() function in future.c looks for __future__ imports and identifies them based on the condition (ds->v.ImportFrom.module == future) where future is constructed using PyString_InternFromString("__future__"). That is, the module attribute of the ImportFrom node is compared by identity with the interned string "__future__".

The AST which I was compiling used the string "__future__" after extracting that string from a much longer string of user input, and as a result, the string was not interned.

The attached file is a simple script demonstrating this issue. I have confirmed that the issue occurs in Python 2.7.2 and 3.2.2.
msg156478 - (view) Author: J. D. Bartlett (talljosh) Date: 2012-03-21 06:23
Incidentally, the workaround that I'm using for the time being is to run the following code before attempting to compile root_node.

for node in ast.walk(root_node):
    if isinstance(node, ast.ImportFrom) and node.module == '__future__':
        node.module = '__future__'
msg156548 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-03-22 12:19
New changeset f57cbcefde34 by Benjamin Peterson in branch '3.2':
check by equality for __future__ not identity (closes #14378)

New changeset 9d793be3b4eb by Benjamin Peterson in branch 'default':
merge 3.2 (#14378)
msg156554 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-03-22 12:49
After this commit the buildbots are dying randomly with segfaults.
msg156555 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-03-22 12:58
New changeset 1729ec440bb6 by Benjamin Peterson in branch '2.7':
check by equality for __future__ not identity (closes #14378)
msg156556 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-03-22 12:58
Date User Action Args
2012-03-22 12:58:47benjamin.petersonsetmessages: + msg156556
2012-03-22 12:58:03python-devsetstatus: open -> closed

messages: + msg156555
2012-03-22 12:49:03r.david.murraysetstatus: closed -> open
priority: normal -> release blocker

nosy: + r.david.murray
messages: + msg156554
2012-03-22 12:19:59python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg156548

resolution: fixed
stage: resolved
2012-03-22 05:18:52eric.araujosetnosy: + brett.cannon, georg.brandl, ncoghlan, benjamin.peterson
2012-03-21 06:23:37talljoshsetmessages: + msg156478
2012-03-21 06:19:41talljoshcreate