classification
Title: ast.dump() with incomplete node
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: miss-islington, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2019-08-26 07:35 by serhiy.storchaka, last changed 2019-08-29 08:12 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 15510 merged serhiy.storchaka, 2019-08-26 07:48
PR 15581 merged miss-islington, 2019-08-29 06:30
PR 15582 merged serhiy.storchaka, 2019-08-29 06:37
Messages (4)
msg350506 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-08-26 07:35
There are several issues in ast.dump() with incompletely initialized node. Some fields and attributes of AST nodes are optional, but creating an AST node without them leads ast.dump() to fail or to produce incorrect result.

1. With annotate_fields=False ast.dump() outputs subnodes as positional arguments of the node constructor.

>>> ast.dump(ast.Raise(exc=ast.Name(id='a', ctx=ast.Load()), cause=ast.Name(id='b', ctx=ast.Load())), annotate_fields=False)
"Raise(Name('a', Load()), Name('b', Load()))"

But if miss the optional exc field it outputs incorrect output:

>>> ast.dump(ast.Raise(cause=ast.Name(id='a', ctx=ast.Load())), annotate_fields=False)
"Raise(Name('a', Load()))"

which is not distinguished from the case when you pass only the exc field and miss the cause field (both are optional):

>>> ast.dump(ast.Raise(exc=ast.Name(id='a', ctx=ast.Load())), annotate_fields=False)
"Raise(Name('a', Load()))"

2. The documentation of ast.dump() says that its result with annotate_fields=True is impossible to evaluate, but this is not true, because keyword arguments are supported by AST node constructors.

3. Attributes end_lineno and end_col_offset are optional, but if you miss them ast.dump() with include_attributes=True will fail:

>>> ast.dump(ast.Raise(lineno=3, col_offset=4), include_attributes=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython3.8/Lib/ast.py", line 126, in dump
    return _format(node)
  File "/home/serhiy/py/cpython3.8/Lib/ast.py", line 118, in _format
    rv += ', '.join('%s=%s' % (a, _format(getattr(node, a)))
  File "/home/serhiy/py/cpython3.8/Lib/ast.py", line 118, in <genexpr>
    rv += ', '.join('%s=%s' % (a, _format(getattr(node, a)))
AttributeError: 'Raise' object has no attribute 'end_lineno'

4. Even if you specify all attributes, the output looks weird if you do not specify any field (note a space after "("):

>>> ast.dump(ast.Raise(lineno=3, col_offset=4, end_lineno=3, end_col_offset=24), include_attributes=True)
'Raise( lineno=3, col_offset=4, end_lineno=3, end_col_offset=24)'
msg350732 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-08-29 06:30
New changeset e64f948e762a6b9fd02e2902ccf42438df6fcb61 by Serhiy Storchaka in branch 'master':
bpo-37950: Fix ast.dump() when call with incompletely initialized node. (GH-15510)
https://github.com/python/cpython/commit/e64f948e762a6b9fd02e2902ccf42438df6fcb61
msg350740 - (view) Author: miss-islington (miss-islington) Date: 2019-08-29 07:04
New changeset d3d2650cf84e6be0f84d6d0d538999d1e0cfdd43 by Miss Islington (bot) in branch '3.7':
bpo-37950: Fix ast.dump() when call with incompletely initialized node. (GH-15510)
https://github.com/python/cpython/commit/d3d2650cf84e6be0f84d6d0d538999d1e0cfdd43
msg350745 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-08-29 07:50
New changeset 097eae5b9b4801d34bb900c01b5e6a80f028bc12 by Serhiy Storchaka in branch '3.8':
[3.8] bpo-37950: Fix ast.dump() when call with incompletely initialized node. (GH-15510) (GH-15582)
https://github.com/python/cpython/commit/097eae5b9b4801d34bb900c01b5e6a80f028bc12
History
Date User Action Args
2019-08-29 08:12:59serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-08-29 07:50:37serhiy.storchakasetmessages: + msg350745
2019-08-29 07:04:47miss-islingtonsetnosy: + miss-islington
messages: + msg350740
2019-08-29 06:37:16serhiy.storchakasetpull_requests: + pull_request15258
2019-08-29 06:30:39miss-islingtonsetpull_requests: + pull_request15257
2019-08-29 06:30:26serhiy.storchakasetmessages: + msg350732
2019-08-26 07:48:37serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request15196
2019-08-26 07:35:54serhiy.storchakacreate