classification
Title: rlcompleter.Completer matches too much
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Claudiu.Popa, donlorenzo, martin.panter, r.david.murray, rbcollins
Priority: normal Keywords: patch

Created on 2014-08-05 15:59 by donlorenzo, last changed 2016-05-13 12:11 by martin.panter.

Files
File name Uploaded Description Edit
rlcompleter.diff donlorenzo, 2014-08-05 16:18 change regular expression to capture the entire attribute
rlcompleter_22141.patch donlorenzo, 2014-10-29 13:37 review
Messages (10)
msg224848 - (view) Author: Lorenz Quack (donlorenzo) * Date: 2014-08-05 15:59
Example:

>>> completer = rlcompleter.Completer()

>>> class A(object):
...     def foo(): pass
...     def foobar(): pass

>>> completer.complete("A.foo(", 0)
'A.foo('
>>> completer.complete("A.foo(", 1)
'A.foobar('

I consider the last match a bug.

The root of this bug is that in attr_matches the regular expression ignores any trailing non-alphanumeric characters by using the "\w" sequence. Note that it would also match "A.foo?%&@" to both "A.foo" and "A.foobar".

I propose this regex instead:
r"(\w+(\.\w+)*)\.([^.]*)"
What do people think?
msg224853 - (view) Author: Lorenz Quack (donlorenzo) * Date: 2014-08-05 16:18
attached the mini patch changing the regular expression in the proposed way
msg227581 - (view) Author: PCManticore (Claudiu.Popa) * (Python triager) Date: 2014-09-26 00:37
Looks good. Could you add a test that reproduces the intended behaviour?
msg227603 - (view) Author: Lorenz Quack (donlorenzo) * Date: 2014-09-26 10:35
Oops!

tests sound like a good Idea.
I realized my "fix" doesn't work.
I had not noticed this before because in my application I had already implemented a workaround :/

The problem with catching the trailing parenthesis is that the group then does not match the attribute of the class.
I'll be back with a new patch and test case.
msg230208 - (view) Author: Lorenz Quack (donlorenzo) * Date: 2014-10-29 13:20
sorry for the delay but here is a new patch with a test.

I changed the strategy for fixing the bug because the dealing with the opening parenthesis became to complicated.

So, now I simply check whether the match actually startswith the search phrase before adding the match to the results. This is like a sanity check which fixes this bug.
msg230267 - (view) Author: PCManticore (Claudiu.Popa) * (Python triager) Date: 2014-10-30 13:01
Looks good to me. You might want to sign the contributor agreement: https://www.python.org/psf/contrib/contrib-form/. This is required for non-trivial contributions. You'll get a * next to your name after signing it.
msg236418 - (view) Author: Lorenz Quack (donlorenzo) * Date: 2015-02-22 22:49
sorry it took so long but the contributor agreement is now signed.
msg247502 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2015-07-28 04:31
I'm struggling to understand this bug. I've tried idle and plain cPython and neither exhibit it. I suspect thats due to how readline is itself tokenizing things.

Python 3.6.0a0 (default:ef5a2ba9df62, Jul 28 2015, 15:48:19) 
[GCC 4.9.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> readline.parse_and_bind("tab: complete")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'readline' is not defined
>>> import readline
>>> readline.parse_and_bind("tab: complete")
>>> class A:
...  def foo(self):pass
...  def foobar(self):pass
... 
>>> A.foo
A.foo(     A.foobar(  
>>> A.foo(      

Whats probably not obvious there is that TAB at the ( indented a tab, rather than offering any completions.

I'm hesitant to apply this without a deeper understanding of where its used, in case of side effects.
msg247601 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-07-29 18:40
I agree with Robert.  You'll have to convince me that this is an actual bug.  It seems reasonable to me that foobar would be returned in this case...it seems to me it is analogous to what my zsh shell does when I hit tab and there's no exact match but there is a match if it interpolates.  Since this doesn't evidence as a bug at the python prompt, there needs to be more of an argument as to why it is wrong...and probably too much risk of breaking working code to change it anyway, absent a bug at the python prompt.

I'm inclined to reject this.
msg265469 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-05-13 12:11
Maybe there are two special cases colliding here:

1. complete("A.foo", 0) returns "A.foo(". But the opening bracket is in readline.get_completer_delims(), which means the completer has suggested not only a “word” but also a delimiter.

2. Perhaps Lorenz has taken "A.foo(" directly and passed it back to the completer. But as Robert said, if Readline were used, it would see the bracket as a delimiter, and only pass an empty string as the text.

The question is how should the completer behave when it gets (invalid) input that would not be possible from default Readline usage?

Lorenz: why are you passing "A.foo(" to the completer, when it only parses attribute names, and not other Python syntax such as brackets?
History
Date User Action Args
2016-05-13 12:11:01martin.pantersetnosy: + martin.panter
messages: + msg265469
2015-07-29 18:42:45rbcollinssetstatus: pending -> open
stage: commit review -> patch review
2015-07-29 18:40:38r.david.murraysetstatus: open -> pending
nosy: + r.david.murray
messages: + msg247601

2015-07-28 04:31:41rbcollinssetnosy: + rbcollins
messages: + msg247502
2015-02-22 22:49:26donlorenzosetmessages: + msg236418
2014-10-30 13:01:41Claudiu.Popasetmessages: + msg230267
stage: patch review -> commit review
2014-10-29 13:37:14donlorenzosetfiles: + rlcompleter_22141.patch
2014-10-29 13:36:21donlorenzosetfiles: - rlcompleter_22141.patch
2014-10-29 13:20:28donlorenzosetfiles: + rlcompleter_22141.patch

messages: + msg230208
2014-09-26 10:35:27donlorenzosetmessages: + msg227603
2014-09-26 00:37:23Claudiu.Popasetnosy: + Claudiu.Popa
messages: + msg227581
2014-08-29 20:40:08terry.reedysetstage: patch review
versions: - Python 3.1, Python 3.2, Python 3.3
2014-08-05 16:18:04donlorenzosetfiles: + rlcompleter.diff
keywords: + patch
messages: + msg224853
2014-08-05 15:59:59donlorenzocreate