Author BTaskaya
Recipients BTaskaya, lys.nikolaou, pablogsal, serhiy.storchaka
Date 2020-10-19.19:58:21
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1603137502.35.0.449039325711.issue42086@roundup.psfhosted.org>
In-reply-to
Content
Simple constructors (basically constructors with no fields) are currently 'cached' / pre-created and dispatched. What I mean by that is, when a user parses some code with `ast.parse()` and gets a tree object, all simple constructors with the same value would point to the same object.

>>> mod1 = ast.parse("a + b", mode="eval")
>>> mod1.body.left.ctx
<ast.Load object at 0x7f8695321d20>
>>> mod1.body.right.ctx
<ast.Load object at 0x7f8695321d20>
>>> mod1.body.left.ctx is mod1.body.right.ctx
True
>>> mod1.body.left.ctx.some_annotation_that_my_extra_process_tool_puts = 1
>>> mod1.body.right.ctx.some_annotation_that_my_extra_process_tool_puts
1

Even though I have no real evidence that, this was done on purpose, I believe this is some sort of 'enum' replication (caching singletons but not really, since this is only valid for the results of ast.parse)
>>> mod1.body.right.ctx is ast.Load()
False
>>> ast.parse("a + b", mode="eval").body.left.ctx is ast.parse("c + d", mode="eval").body.right.ctx
True



Obviously, we can not change these into enums like (ast.expr_ctx.Load / LOAD) since it would break theoretically most of the code that works with ast. So here is a tl;dr:

- Even though all ast objects are mutable (by default), we use the same objects when converting C AST into Python AST. So that mutations on one object is affecting the rest of the tree. 

- We can either;
  - Document this behavior and keep it
  - Return freshly constructed objects when converting C AST into Python AST
    This has a very slight (that I suspect no body uses) risk of breaking code, but as I implied, very slight. Which would occur in such a case

import ast
from collections import defaultdict

def collect_contexts(tree):
    contexts = defaultdict(list)
    for node in ast.walk(tree):
        if isinstance(node, ast.Name):
            contexts[node.ctx].append(node.id)
    return contexts
    
print(collect_contexts(ast.parse("a, b = c, d")))

This code can easily (and it makes it more reliable/robust) refactored into 
-             contexts[node.ctx].append(node.id)
-             contexts[type(node.ctx)].append(node.id)
but just mentioning in case of any question appears about backwards incompatability.
History
Date User Action Args
2020-10-19 19:58:22BTaskayasetrecipients: + BTaskaya, serhiy.storchaka, lys.nikolaou, pablogsal
2020-10-19 19:58:22BTaskayasetmessageid: <1603137502.35.0.449039325711.issue42086@roundup.psfhosted.org>
2020-10-19 19:58:22BTaskayalinkissue42086 messages
2020-10-19 19:58:21BTaskayacreate