This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: AttributeError in ast.unparse
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: BTaskaya, Mark.Shannon, benjamin.peterson, brett.cannon, lukasz.langa, pablogsal, terry.reedy, xiaket, yselivanov
Priority: normal Keywords: patch

Created on 2021-08-12 01:26 by xiaket, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27770 closed BTaskaya, 2021-08-14 17:54
Messages (4)
msg399427 - (view) Author: Kai Xia (xiaket) * Date: 2021-08-12 01:26
I was trying to construct an ast object dynamically and I think I can identify some potential issue.

With the following snippet:

```
#!/usr/bin/env python3
import ast
import sys

print(sys.version)

good = ast.Assign(
    targets=[ast.Name(id="hello", ctx=ast.Store())],
    value=ast.Constant(value="world"),
    lineno=1
)
print(ast.unparse(good))


bad = ast.Assign(
    targets=[ast.Name(id="hello", ctx=ast.Store())],
    value=ast.Constant(value="world"),
)
print(ast.unparse(bad))
```

On my box the output looks like:

```
3.9.6 (default, Jun 29 2021, 05:25:02)
[Clang 12.0.5 (clang-1205.0.22.9)]
hello = 'world'
Traceback (most recent call last):
  File "/Users/xiaket/py.py", line 19, in <module>
    print(ast.unparse(bad))
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ast.py", line 1572, in unparse
    return unparser.visit(ast_obj)
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ast.py", line 801, in visit
    self.traverse(node)
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ast.py", line 795, in traverse
    super().visit(node)
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ast.py", line 407, in visit
    return visitor(node)
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ast.py", line 858, in visit_Assign
    if type_comment := self.get_type_comment(node):
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ast.py", line 786, in get_type_comment
    comment = self._type_ignores.get(node.lineno) or node.type_comment
AttributeError: 'Assign' object has no attribute 'lineno'
```

As I can understand, when we need to construct the Assign object, we'll need to provide two keyword arguments, targets and value. We don't need to provide the `lineno` as it should be an attribute of the statement node. Also, if we don't run `unparse` against the object, apparently it works fine.

I think in the `get_type_comment` method, we are making the assumption that the lineno is set automatically, this is true when we are parsing python source code as string. But when we are creating the object from scratch, we don't have that `lineno` attribute and it will fail.
msg399593 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2021-08-14 14:42
I don't think this is really an issue considering some other functionalities that consumes AST code (e.g `compile`) will expect it to be annotated with location metadata. This is why the `ast` module already comes bundled with a utility function called `fix_missing_locations` which in your example would work;

> print(ast.unparse(bad))
> print(ast.unparse(ast.fix_missing_locations(bad)))

Though I recall seeing this error once before, and since there also some counter examples (e.g ast.increment_lineno) that assume an AST node might lack of these attributes even though the nodes declare them makes it worth to fix.
msg402868 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-09-29 13:17
Sorry, I think we shouldn't be changing `ast.py` this way. To reiterate my review comment from the PR:

I think we should just let this go and instead advertise that if you put new synthetic nodes in a tree, you have to either:
- provide `lineno` and `col_offset` yourself; or
- use `fix_missing_locations()` to fill out the information for you.

The problem with the logic that I commented on twice now (on the PR) is one reason for my opinion. Another is bigger though:

Suppose user code is replacing nodes or just inserting new ones. This operation not only creates nodes without location attributes. It also in all likelihood *moves* locations of type comments and those aren't updated (for example, `_type_ignores[123]` should now really be `_type_ignores[125]` to still point at the same tokens). So user code shouldn't be naive when it comes to missing locations, and we definitely shouldn't paper over missing locations.
msg408824 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-12-17 23:50
#46073 may be a duplicate of this and perhaps should have the same resolution.
History
Date User Action Args
2022-04-11 14:59:48adminsetgithub: 89059
2022-03-19 15:01:30iritkatriellinkissue46073 superseder
2021-12-17 23:50:34terry.reedysetnosy: + terry.reedy
messages: + msg408824
2021-12-14 22:58:02BTaskayasetstatus: open -> closed
2021-09-29 13:17:09lukasz.langasetnosy: + lukasz.langa
messages: + msg402868

resolution: rejected
stage: patch review -> resolved
2021-08-16 08:04:15lukasz.langasetversions: + Python 3.10, Python 3.11
2021-08-14 17:54:18BTaskayasetkeywords: + patch
stage: patch review
pull_requests: + pull_request26245
2021-08-14 14:42:22BTaskayasetmessages: + msg399593
2021-08-13 23:05:25terry.reedysettitle: Issue with unparse in ast module -> AttributeError in ast.unparse
2021-08-13 23:04:23terry.reedysetnosy: + brett.cannon, benjamin.peterson, Mark.Shannon, yselivanov, pablogsal, BTaskaya
2021-08-12 01:26:20xiaketcreate