This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: `from __future__ import annotations` has no effect inside `ast.parse`
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: lukasz.langa Nosy List: BTaskaya, benjamin.peterson, eric.smith, kayhayen, levkivskyi, lukasz.langa, pablogsal
Priority: normal Keywords:

Created on 2018-11-02 08:28 by kayhayen, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (7)
msg329114 - (view) Author: Kay Hayen (kayhayen) Date: 2018-11-02 08:28
Hello,

in trying to know what to put into "__annotations__" at run time, the "from __future__ import annotations" pose a new problem. It is now necessary to "unparse" to ast code that is still there. 

Code to do so, is in C API "_PyAST_ExprAsUnicode", but won't work with the Python level ast objects.

I have a kludge, using "compile" and "exec", to get past that. It's pretty ugly to do it like this, and Nuitka cannot be alone in trying to predict the value of "__annotations__". 

This code is my "ast.unparse" replacement:

def unparse(node):
        _host_node = ast.parse("x:1")

        _host_node.body[0].annotation = node

        r = compile(_host_node, "<annotations>", "exec", 1048576, dont_inherit = True)

        # Using exec here, to compile the ast node tree back to string,
        # there is no accessible "ast.unparse", and this works as a hack
        # to convert our node to a string annotation, pylint: disable=exec-used
        m = {}
        exec(r, m) 

        return m["__annotations__"]['x']

I am caching "_host_node" in the real code.

Having a real "ast.unparse" would be better however. It seems that the building blocks are all there, just not in that form.

Yours,
Kay
msg329904 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2018-11-14 12:46
I think this is a bug, actually. It's going to be an incompatibility in `ast.parse` so we'll probably want to only enable it in Python 3.8.

This should have stringified annotations:

>>> p = ast.parse("""
... from __future__ import annotations
... a: 1
... """)
>>> p.body[1].annotation  # ACTUAL
<_ast.Num object at 0x1048cf470>

>>> p.body[1].annotation  # EXPECTED
<_ast.Str object at 0x1048cf6a0>
msg356827 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2019-11-17 18:58
Hey @lukasz.langa, I want to work on this. Should we add an interface to _PyAST_ExprAsUnicode or just the bugfix for annotations?
msg358907 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2019-12-27 17:55
> Having a real "ast.unparse" would be better however. It seems that the building blocks are all there, just not in that form.

Now we have that :) https://docs.python.org/3.9/whatsnew/3.9.html#improved-modules
msg358910 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-12-27 20:05
I don't think this is a bug. Technically the 'from __future__ import annotations' is a compiler option, not an AST one. Also, the flag says that "annotations become strings at runtime" and at the point, we have an AST is not still runtime. The CO_FUTURE_ANNOTATIONS only takes place in the compiler that reverses the AST object into a string. If we make the AST aware of CO_FUTURE_ANNOTATIONS, then what the compiler will see in the presence of  'from __future__ import annotations' would be different if you compile a module than if you parse it using ast.parse() and then compile the result (in the second case the compiler will see strings in the annotations, not AST nodes while in the first case it will see nodes). Additionally, doing this will limit tools that want to analyze the contents of those annotations before they get turned into strings (like type checker, linters...etc).
msg358941 - (view) Author: Kay Hayen (kayhayen) Date: 2019-12-28 11:49
Thanks a lot, Batuhan, this will feel a lot more correct, from my point of view, this could be closed unless there would be a backport.
msg358945 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2019-12-28 14:04
I agree with Pablo's analysis. And this can't be backported, since it's a new feature. So I'm closing this issue.
History
Date User Action Args
2022-04-11 14:59:07adminsetgithub: 79324
2019-12-28 14:04:14eric.smithsetstatus: open -> closed
resolution: fixed
messages: + msg358945

stage: needs patch -> resolved
2019-12-28 11:49:29kayhayensetmessages: + msg358941
2019-12-27 20:05:11pablogsalsetnosy: + pablogsal
messages: + msg358910
2019-12-27 17:55:02BTaskayasetmessages: + msg358907
2019-11-17 18:58:49BTaskayasetnosy: + BTaskaya, benjamin.peterson
messages: + msg356827
2018-11-14 12:46:34lukasz.langasettitle: Annotations future requires unparse, but not accessible from Python -> `from __future__ import annotations` has no effect inside `ast.parse`
messages: + msg329904

assignee: lukasz.langa
type: enhancement -> behavior
stage: needs patch
2018-11-06 07:22:36eric.smithsetnosy: + eric.smith
2018-11-02 20:50:19levkivskyisetnosy: + levkivskyi
2018-11-02 11:22:12serhiy.storchakasetnosy: + lukasz.langa

components: + Extension Modules
versions: + Python 3.8, - Python 3.7
2018-11-02 09:23:37xtreaksettitle: Annotations future requires unparse, but not accessible from Pythpn -> Annotations future requires unparse, but not accessible from Python
2018-11-02 08:28:53kayhayencreate