classification
Title: ast.Call end_lineno is defined and returns None
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Anthony Sottile, BTaskaya, gaborbernat, laloch, miss-islington, pablogsal, remi.lapeyre, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2020-05-22 08:34 by gaborbernat, last changed 2020-08-05 19:34 by pablogsal. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 20312 merged BTaskaya, 2020-05-22 09:47
PR 21741 merged miss-islington, 2020-08-05 13:33
PR 21742 closed pablogsal, 2020-08-05 13:35
PR 21745 merged BTaskaya, 2020-08-05 18:21
Messages (13)
msg369567 - (view) Author: gaborbernat (gaborbernat) * Date: 2020-05-22 08:34
Reporting an issue from https://github.com/xonsh/xonsh/issues/3581; boils down to, ast.Call used to not define end_lineno in 3.8:

py -3.8 -c 'import ast; type(ast.Call().end_lineno)'                                                                                                                                                                 Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'Call' object has no attribute 'end_lineno'

However in 3.9 is defined with None:

py -3.9 -c 'import ast; type(ast.Call().end_lineno)'

This messes with some other operations in ast, namely https://github.com/python/cpython/blob/master/Lib/ast.py#L233
msg369569 - (view) Author: David Strobach (laloch) Date: 2020-05-22 08:49
The issue is not limited to ast.Call. Other AST nodes are also affected (e.g. ast.BinOp). It's also not limited to end_lineno attribute. The same applies to end_col_offset.
msg369572 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python triager) Date: 2020-05-22 09:28
This is because the 'end_lineno' and 'end_col_offset' are declared as optional attributes in the ASDL spec. The commit 'b7e9525f9c7ef02a1d2ad8253afdeb733b0951d4' made all optional fields and attributes auto initalized with None.
msg369574 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python triager) Date: 2020-05-22 09:29
See here for the complete spec: https://docs.python.org/3.9/library/ast.html#abstract-grammar
msg369577 - (view) Author: Rémi Lapeyre (remi.lapeyre) * Date: 2020-05-22 09:39
This was done in b7e9525f9c7ef02a1d2ad8253afdeb733b0951d4 for issue 36287, all attribute should now be defined, even when they are not set. It looks like some parts of the ast module where not updated.

If nobody works on this I will send a PR to update the rest of AST later today.
msg369581 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python triager) Date: 2020-05-22 09:49
> If nobody works on this I will send a PR to update the rest of AST later today.

:/ I'm terribly sorry for that, I just sent a PR without refreshing the tab.
msg369614 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2020-05-22 16:07
There's current expectation in a lot of linters / code formatters that the *lineno and *col_offset attributes will be missing if they are not attached to the node

setting them to None is going to break a lot of those, if possible I'd suggest going back to when they were missing
msg369619 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python triager) Date: 2020-05-22 17:18
> setting them to None is going to break a lot of those, if possible I'd suggest going back to when they were missing

'lineno' and 'col_offset' will never be none, since both are declared as normal integers. But on the other hand, 'end_lineno' and 'end_col_offset' are declared as optional integers which would make sense to auto initalize them with *None*, and I dont think it would ever break some code that complies with the ASDL declaration.
msg369621 - (view) Author: David Strobach (laloch) Date: 2020-05-22 17:40
Actually, Xonsh (http://github.com/xonsh/xonsh) tests show that keyword AST nodes are missing 'lineno' attribute, but that could be our fault.
msg369623 - (view) Author: David Strobach (laloch) Date: 2020-05-22 18:07
> Actually, Xonsh tests show that keyword AST nodes are missing 'lineno' attribute, but that could be our fault.

Yes, our fault. Sorry for the noise.
msg374880 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-08-05 13:33
New changeset 8f4380d2f5839a321475104765221a7394a9d649 by Batuhan Taskaya in branch 'master':
bpo-40726: handle uninitalized end_lineno on ast.increment_lineno (GH-20312)
https://github.com/python/cpython/commit/8f4380d2f5839a321475104765221a7394a9d649
msg374882 - (view) Author: miss-islington (miss-islington) Date: 2020-08-05 13:52
New changeset a1320989f5350439e0677450f49b36f2c10583d2 by Miss Islington (bot) in branch '3.9':
bpo-40726: handle uninitalized end_lineno on ast.increment_lineno (GH-20312)
https://github.com/python/cpython/commit/a1320989f5350439e0677450f49b36f2c10583d2
msg374893 - (view) Author: miss-islington (miss-islington) Date: 2020-08-05 18:37
New changeset b24c9d2b0656764bef48120d9511faf833bd7ead by Batuhan Taskaya in branch '3.8':
[3.8] bpo-40726: handle uninitalized end_lineno on ast.increment_lineno (GH-21745)
https://github.com/python/cpython/commit/b24c9d2b0656764bef48120d9511faf833bd7ead
History
Date User Action Args
2020-08-05 19:34:48pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-08-05 18:37:26miss-islingtonsetmessages: + msg374893
2020-08-05 18:21:15BTaskayasetpull_requests: + pull_request20889
2020-08-05 13:52:37miss-islingtonsetmessages: + msg374882
2020-08-05 13:35:45pablogsalsetpull_requests: + pull_request20888
2020-08-05 13:33:03miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request20887
2020-08-05 13:33:00pablogsalsetmessages: + msg374880
2020-05-23 16:31:15DahlitzFloriansetnosy: - DahlitzFlorian
2020-05-22 18:07:16lalochsetmessages: + msg369623
2020-05-22 17:40:37lalochsetmessages: + msg369621
2020-05-22 17:18:32BTaskayasetmessages: + msg369619
2020-05-22 16:14:39pablogsalsetnosy: + serhiy.storchaka
2020-05-22 16:07:14Anthony Sottilesetnosy: + Anthony Sottile
messages: + msg369614
2020-05-22 09:49:10BTaskayasetmessages: + msg369581
2020-05-22 09:47:50BTaskayasetkeywords: + patch
nosy: + BTaskaya

pull_requests: + pull_request19581
stage: patch review
2020-05-22 09:41:59DahlitzFloriansetnosy: + DahlitzFlorian
2020-05-22 09:39:49remi.lapeyresetversions: + Python 3.10
nosy: + remi.lapeyre, - serhiy.storchaka, BTaskaya

messages: + msg369577

components: + Library (Lib)
type: behavior
2020-05-22 09:30:56BTaskayasetnosy: + serhiy.storchaka
2020-05-22 09:29:45BTaskayasetmessages: + msg369574
2020-05-22 09:28:27BTaskayasetmessages: + msg369572
2020-05-22 09:22:56BTaskayasetnosy: + BTaskaya
2020-05-22 08:49:11lalochsetmessages: + msg369569
2020-05-22 08:41:48lalochsetnosy: + laloch
2020-05-22 08:38:40xtreaksetnosy: + pablogsal
2020-05-22 08:34:59gaborbernatcreate