This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ast identifies incorrect column for compound attribute lookups
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Justin McCann, andrewbriand, levkivskyi, serhiy.storchaka
Priority: normal Keywords:

Created on 2019-09-13 21:48 by Justin McCann, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (4)
msg352393 - (view) Author: Justin McCann (Justin McCann) Date: 2019-09-13 21:48
This issue is related to https://github.com/microsoft/vscode-python/issues/7327 and https://github.com/PyCQA/pylint/issues/3103

For compound attribute access in variable references like a.b.c.d, the AST reports the column of the first variable/attribute in the sequence instead of the specific attribute location. For example, the location of  c is reported as column 0 (a) and not column 4 (c).

Here's the AST test case provided by a pylint developer in a comment on pylint issue 3103; I confirmed the behavior is the same in python 3.8.0b4.
```
$ python3.8
Python 3.8.0b4 (v3.8.0b4:d93605de72, Aug 29 2019, 21:47:47)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> body = ast.parse('''
... print(x.item.akey > 2)
... ''')
>>> # x.item
... x_item = body.body[0].value.args[0].left.value
>>> print(x_item, x_item.attr, x_item.col_offset)
<_ast.Attribute object at 0x10a7751f0> item 6
>>> # x.item.akey
... print(x_item.value, x_item.value.col_offset) .  ### probably should be 8
<_ast.Name object at 0x10a775280> 6
```

Related issues:
* https://bugs.python.org/issue1440601 Add col information to parse & ast nodes
* https://bugs.python.org/issue10769 ast: provide more useful range information


Here is the resulting confusion when you use this output in pylint (and then VSCode highlights only "x" since it's the variable that starts in column 0):

Original pylint/vscode testcase:
```
class TestMe:
    def __init__(self):
        self.item = {'akey': 42}
        self.again = self

x = TestMe()
### pylint error message here is
###    testme.py:11:6: E1101: Instance of 'dict' has no 'akey' member (no-member)
### The problem is with `x.item`, but pylint shows the column for `x`
print(x.item.akey > 2)

print(x.again.item.doesnotexist)
```

Current behavior
$ pylint testme.py  -rn -sn
************* Module testme
testme.py:10:6: E1101: Instance of 'dict' has no 'akey' member (no-member)
testme.py:12:6: E1101: Instance of 'dict' has no 'doesnotexist' member (no-member)

Expected behavior
$ pylint testme.py  -rn -sn
************* Module testme
testme.py:10:8: E1101: Instance of 'dict' has no 'akey' member (no-member)
testme.py:12:14: E1101: Instance of 'dict' has no 'doesnotexist' member (no-member)

$ pylint --version output
pylint 2.3.1
astroid 2.2.5
Python 3.7.4 (default, Jul 9 2019, 18:13:23)
[Clang 10.0.1 (clang-1001.0.46.4)]
msg352466 - (view) Author: Andrew Briand (andrewbriand) Date: 2019-09-15 07:37
It looks like the test suite (in particular test_ast) specifically checks for the behavior where the col_offset of c in a.b.c.d is 0. This  seems strange to me though, does anyone know if this is intended? If not, I can patch it.
msg352467 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-09-15 08:36
AST for expression a.b.c is

         Attribute(
            value=Attribute(
               value=Name(
                  id='a',
                  ctx=Load(),
                  lineno=1,
                  col_offset=0,
                  end_lineno=1,
                  end_col_offset=1),
               attr='b',
               ctx=Load(),
               lineno=1,
               col_offset=0,
               end_lineno=1,
               end_col_offset=3),
            attr='c',
            ctx=Load(),
            lineno=1,
            col_offset=0,
            end_lineno=1,
            end_col_offset=5)

It contains one Name node for "a" and two Attribute nodes for "a.b" and "a.b.c". Every node has attributes lineno, col_offset, end_lineno and end_col_offset. Note that there are no nodes for "b" and "c", therefore there is no information about the location of "c" in AST.
msg352941 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2019-09-21 15:04
As Serhiy explained, there is no bug here, _potentially_ we could change AST so that attribute name (i.e. "b" in "a.b") is wrapped in a special node holding the position info (similar to ``ast.arg``), but it is a breaking change and is not worth it. You can re-open your issue on PyLint tracker.
History
Date User Action Args
2022-04-11 14:59:20adminsetgithub: 82347
2019-09-21 15:04:03levkivskyisetstatus: open -> closed

nosy: + levkivskyi
messages: + msg352941

resolution: not a bug
stage: resolved
2019-09-15 08:36:39serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg352467
2019-09-15 07:37:56andrewbriandsetnosy: + andrewbriand
messages: + msg352466
2019-09-13 21:48:54Justin McCanncreate