classification
Title: Tab completion executes @property getter function
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: RPecor, jack__d, lukasz.langa, miss-islington
Priority: normal Keywords: patch

Created on 2021-07-27 21:41 by RPecor, last changed 2021-07-29 15:48 by lukasz.langa. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27401 merged jack__d, 2021-07-28 03:02
PR 27433 merged jack__d, 2021-07-28 22:56
PR 27444 merged miss-islington, 2021-07-29 11:40
PR 27445 merged miss-islington, 2021-07-29 11:40
PR 27446 merged miss-islington, 2021-07-29 14:01
PR 27447 merged miss-islington, 2021-07-29 14:01
Messages (14)
msg398323 - (view) Author: Ryan Pecor (RPecor) Date: 2021-07-27 21:41
After making a class using the @property decorator to implement a getter, using tab completion that matches the getter function name executes the function. 

See below for example (line numbers added, <tab> indicates when the user presses the tab key):

1  >>> class Foo(object):
2  ...     def __init__(self,value):
3  ...         self.value = value
4  ...     @property
5  ...     def print_value(self):
6  ...         print("Foo has a value of {}".format(self.value))
7  ... 
8  >>> bar = Foo(4)
9  >>> bar.<tab>~~~Foo has a value of 4~~~
10 <tab>~~~Foo has a value of 4~~~
11 
12 bar.print_value  bar.value        
13 >>> bar.p<tab>~~~Foo has a value of 4~~~
14 <tab>rint_value~~~Foo has a value of 4~~~
15 ~~~Foo has a value of 4~~~
16 
17 bar.print_value
18 >>> bar.v<tab>alue

I pressed tab after typing "bar." in line 9. It then printed the remainder of line 9 and moved the cursor to line 10. Pressing tab again prints line 10 and 11 before finally showing the expected output on line 12. lines 13-17 follow the same steps, but after typing "bar.p" to show that it happens whenever you tab and it matches the getter. Line 18 shows a correct tab completion resulting from hitting tab after typing "bar.v" which does not match the getter function.
msg398324 - (view) Author: Ryan Pecor (RPecor) Date: 2021-07-27 21:54
I forgot to mention that I also added "~~~" to either side of the printed string every time it printed to help differentiate the printed string from commands that I typed into the interpreter.
msg398327 - (view) Author: Jack DeVries (jack__d) * Date: 2021-07-27 22:39
Woah, til the python shell has tab completion! This does seem like undesirable behavior. I'd like to work on a fix for this if that's all right, assuming that this behavior should not occur.

I haven't exactly found where the tab autocomplete is implemented, but I'm assuming I'll find in one of the functions I see in the backtrace when I pause python at the interpreter prompt; maybe one of these? (my best quick guesses by function name).

- _PyRun_InteractiveLoopObject
- PyRun_InteractiveOneObjectEx
- interactive_rule
- statement_newline_rule

If anyone can point me in the right direction, that'd be great, and I think I can work on this one tomorrow if that's all right!
msg398330 - (view) Author: Ryan Pecor (RPecor) Date: 2021-07-27 23:31
It looks to me like the issue is caused by the eval() in line 155 of the rlcompleter.py file (https://github.com/python/cpython/blob/bb3e0c240bc60fe08d332ff5955d54197f79751c/Lib/rlcompleter.py#L155) which runs the function in order to see if it runs or raises an exception.

I thought maybe replacing it with hasattr() might work, but it looks like the issue is repeated there as well!

>>> hasattr(bar, "print_value")
Foo has a value of 4
True

This goes over to the C side of things now (https://github.com/python/cpython/blob/196998e220d6ca030e5a1c8ad63fcaed8e049a98/Python/bltinmodule.c#L1162) and I'll put in another issue regarding that!
msg398331 - (view) Author: Ryan Pecor (RPecor) Date: 2021-07-27 23:58
Actually, hasattr() specifically states that it uses getattr() so that behavior is expected from getattr() so I will not be creating a separate issue for that.

Now that I see hasattr() uses getattr(), it looks like the tab completion issue might not stem from line 155, but from line 180 (https://github.com/python/cpython/blob/bb3e0c240bc60fe08d332ff5955d54197f79751c/Lib/rlcompleter.py#L180) where it calls getattr().

A possible fix to the tab completion issue might be to add to/expand the warning about evaluating arbitrary C code (https://github.com/python/cpython/blob/bb3e0c240bc60fe08d332ff5955d54197f79751c/Lib/rlcompleter.py#L145) when using tab to autocomplete.
msg398341 - (view) Author: Jack DeVries (jack__d) * Date: 2021-07-28 03:12
> Now that I see hasattr() uses getattr(), it looks like the tab completion issue might not stem from line 155, but from line 180 (https://github.com/python/cpython/blob/bb3e0c240bc60fe08d332ff5955d54197f79751c/Lib/rlcompleter.py#L180) where it calls getattr().

That is correct! I was able to fix the undesirable behavior by adding an early exit condition if we appear to have a property object. I checked for the existence of a property object like this:

    class Foo:

        @property
        def bar(self):
            return 'baz'

    def is_property(object, attr):
        return isinstance(getattr(type(object), attr, None), property)

    assert is_property(Foo(), 'bar')

The code that follows line 180 in the loop just checks if ``object.attribute`` is callable, and whether the callable takes arguments in order to determine whether to add parenthesis to the completion. In my opinion, we never really want to add parenthesis when providing completion for a property attribute anyway, so there's no reason to go through that code block if we are dealing with a property.

I opened a GitHub PR. Let me know what you think!
msg398343 - (view) Author: Ryan Pecor (RPecor) Date: 2021-07-28 04:33
Wow, that was quick and the code looks clean too! Thanks for fixing that up!
msg398479 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-29 11:40
New changeset 50de8f74f8e92b20e76438c22b6a8f91afd6df75 by Jack DeVries in branch 'main':
bpo-44752: Make rlcompleter not call `@property` methods (GH-27401)
https://github.com/python/cpython/commit/50de8f74f8e92b20e76438c22b6a8f91afd6df75
msg398483 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-29 12:07
New changeset d20f1095a6a51ee8f41ef445a009d26256e1fa61 by Miss Islington (bot) in branch '3.10':
bpo-44752: Make rlcompleter not call `@property` methods (GH-27401) (GH-27444)
https://github.com/python/cpython/commit/d20f1095a6a51ee8f41ef445a009d26256e1fa61
msg398485 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-29 13:53
New changeset acaf3b959492e7a84dcc9d46101ed6dc313768c8 by Miss Islington (bot) in branch '3.9':
bpo-44752: Make rlcompleter not call `@property` methods (GH-27401) (#27445)
https://github.com/python/cpython/commit/acaf3b959492e7a84dcc9d46101ed6dc313768c8
msg398486 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-29 14:01
New changeset 6741794dd420c6b9775a188690dbf265037cd69f by Jack DeVries in branch 'main':
bpo-44752: refactor part of rlcompleter.Completer.attr_matches (GH-27433)
https://github.com/python/cpython/commit/6741794dd420c6b9775a188690dbf265037cd69f
msg398490 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-29 15:44
New changeset f8e13e35d12bee6e3447c791199522249ba7f05a by Miss Islington (bot) in branch '3.10':
bpo-44752: refactor part of rlcompleter.Completer.attr_matches (GH-27433) (GH-27447)
https://github.com/python/cpython/commit/f8e13e35d12bee6e3447c791199522249ba7f05a
msg398491 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-29 15:46
New changeset 93d90871d216625b8d9db0fa59693953353d19ae by Miss Islington (bot) in branch '3.9':
bpo-44752: refactor part of rlcompleter.Completer.attr_matches (GH-27433) (GH-27446)
https://github.com/python/cpython/commit/93d90871d216625b8d9db0fa59693953353d19ae
msg398492 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-29 15:48
Thanks, Jack! ✨ 🍰 ✨
History
Date User Action Args
2021-07-29 15:48:38lukasz.langasetstatus: open -> closed
versions: + Python 3.10, Python 3.11
messages: + msg398492

resolution: fixed
stage: patch review -> resolved
2021-07-29 15:46:11lukasz.langasetmessages: + msg398491
2021-07-29 15:44:54lukasz.langasetmessages: + msg398490
2021-07-29 14:01:47miss-islingtonsetpull_requests: + pull_request25976
2021-07-29 14:01:42miss-islingtonsetpull_requests: + pull_request25975
2021-07-29 14:01:35lukasz.langasetmessages: + msg398486
2021-07-29 13:53:51lukasz.langasetmessages: + msg398485
2021-07-29 12:07:09lukasz.langasetmessages: + msg398483
2021-07-29 11:40:45miss-islingtonsetpull_requests: + pull_request25974
2021-07-29 11:40:39miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25973
2021-07-29 11:40:33lukasz.langasetnosy: + lukasz.langa
messages: + msg398479
2021-07-28 22:56:12jack__dsetpull_requests: + pull_request25962
2021-07-28 04:33:39RPecorsetmessages: + msg398343
2021-07-28 03:12:33jack__dsetmessages: + msg398341
2021-07-28 03:02:36jack__dsetkeywords: + patch
stage: patch review
pull_requests: + pull_request25934
2021-07-27 23:58:43RPecorsetmessages: + msg398331
2021-07-27 23:31:29RPecorsetmessages: + msg398330
2021-07-27 22:39:20jack__dsetnosy: + jack__d
messages: + msg398327
2021-07-27 21:54:40RPecorsetmessages: + msg398324
2021-07-27 21:41:47RPecorcreate