classification
Title: ast.Tuple has wrong col_offset
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: isaiah, lukasz.langa, r.david.murray
Priority: normal Keywords:

Created on 2018-05-15 15:16 by isaiah, last changed 2018-05-16 08:26 by isaiah. This issue is now closed.

Messages (8)
msg316665 - (view) Author: Isaiah Peng (isaiah) Date: 2018-05-15 15:16
The `col_offset` of the ast.Tuple node is set to the column offset of the first element, shown in code:

>>> a = "{1,2,3}"
>>> b = ast.parse(a).body[0]                                                                                                                                                                  
>>> b.value.col_offset
0
>>> a = "[1,2,3]"                                                                                                                                                                             
>>> b = ast.parse(a).body[0]                                                                                                                                                                  
>>> b.value.col_offset                                                                                                                                                                        
0                                                                                                                                                                                             
>>> a = "(1,2,3)"                                                                                                                                                                             
>>> ast.parse(a).body[0].value.col_offset                                                                                                                                                     
1 
>>> a = "()"                                                                                                                                                                                  
>>> ast.parse(a).body[0].value.col_offset                                                                                                                                                     
0  

It's correct for dict, set, list, even empty tuple, Though this is not a serious bug, for other python implementations that uses the tests as language spec, this is annoying.
msg316684 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2018-05-15 17:33
This is because technically parentheses aren't part of the tuple.  They are just organizational and unnecessary for the tuple to be recognized by the parser.

Those two are equivalent:

  >>> ast.parse("(1,2,3)").body[0].value.col_offset
  1
  >>> ast.parse("(1)").body[0].value.col_offset
  1

You can see similar behavior within generator expressions in contexts where the parentheses are not semantically required:

  >>> ast.parse("c(i for i in range(10))").body[0].value.args[0].col_offset
  2
  >>> ast.parse("c((i for i in range(10)))").body[0].value.args[0].col_offset
  3
msg316685 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2018-05-15 17:33
For comparison, a tuple without parentheses:

  >>> ast.parse("1,2,3").body[0].value.col_offset
  0
msg316708 - (view) Author: Isaiah Peng (isaiah) Date: 2018-05-15 19:45
Thanks for the reply, that's quite reasonable, especially take the generator expression case into consideration. However I found this is not consistent with empty tuple:

>>> a = "()"                                                                                                                                                                                  
>>> ast.parse(a).body[0].value.col_offset                                                                                                                                                     
0

It's true that the parenthesis is required to construct a tuple, but if the parenthesis is served as the starting point of the tuple, then the col_offset should be the opening parenthesis. i.e. in the following example, both should start from col 2:

  >>> ast.parse("c(i for i in range(10))").body[0].value.args[0].col_offset
  2
  >>> ast.parse("c((i for i in range(10)))").body[0].value.args[0].col_offset
  3
msg316710 - (view) Author: Isaiah Peng (isaiah) Date: 2018-05-15 19:46
> It's true that the parenthesis is required to construct a tuple

Sorry, I mean the parenthesis is *not* required.
msg316713 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2018-05-15 20:08
No, the parenthesis are never part of the tuple itself, even if you can't write syntactically correct code without them. They just syntactically group the expression list to isolate it from the surrounding context.  It's the same principle as having an expression like (123).  That's an integer that happens to be surrounded by parenthesis.  The integer itself still starts at column 1.
msg316715 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2018-05-15 20:10
Oh, and the empty tuple is a specific syntactic construct that really is the empty parenthesis, so that's consistent with the language definition.
msg316785 - (view) Author: Isaiah Peng (isaiah) Date: 2018-05-16 08:26
Fair enough, thanks for clarification.
History
Date User Action Args
2018-05-16 08:26:54isaiahsetmessages: + msg316785
2018-05-15 20:10:54r.david.murraysetmessages: + msg316715
2018-05-15 20:08:58r.david.murraysetstatus: open -> closed
resolution: not a bug
stage: resolved
2018-05-15 20:08:26r.david.murraysetnosy: + r.david.murray
messages: + msg316713
2018-05-15 19:46:50isaiahsetmessages: + msg316710
2018-05-15 19:45:02isaiahsetmessages: + msg316708
2018-05-15 17:33:56lukasz.langasetmessages: + msg316685
2018-05-15 17:33:18lukasz.langasetnosy: + lukasz.langa
messages: + msg316684
2018-05-15 15:16:22isaiahcreate