classification
Title: Improve doc for readline.set_completer_delims()
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: 6953 Superseder:
Assigned To: docs@python Nosy List: docs@python, eric.araujo, hideaki_t, martin.panter, python-dev, rheise
Priority: normal Keywords: easy

Created on 2010-12-30 13:08 by rheise, last changed 2016-04-05 22:23 by martin.panter. This issue is now closed.

Messages (7)
msg124917 - (view) Author: rheise (rheise) Date: 2010-12-30 13:08
Python's readline library generates out of the choices provided by a custom completion function the wrong terminal input. Say, the completion function suggests 'foobar' and 'foobaz' as matching completion strings, readline should produce the characters 'fooba' to prompt and await further interaction by the user. 
This is the intended behaviour of readline and this works in Python as well. However this does nod work anymore as soon, as the suggestion 
list contains dashes `-' as input argument. 

A working as supposed example:

>>> import readline
>>> 
>>> def complete(text, state):
...   if (state == 0):
...     return "abc"
...   if (state == 1):
...     return "ade"
...   else:
...     return None
... 
>>> 
>>> readline.parse_and_bind("Tab: complete")
>>> readline.set_completer(complete)
>>> 
>>> raw_input()
a
abc  ade  
a
'a'
>>> 

remark: I entered a and hit tab. readline produces abc/ade as valid choices, stopping at the first ambiguous character. Now consider the following example:

>>> import readline
>>> 
>>> def complete(text, state):
...   if (state == 0):
...     return "a-bc"
...   if (state == 1):
...     return "a-de"
...   else:
...     return None
... 
>>> 
>>> readline.parse_and_bind("Tab: complete")
>>> readline.set_completer(complete)
>>> 
>>> raw_input()
a-a-a-
'a-a-a-'
>>> 

The intended behaviour is the very same as for the first example. Readline should produce 'a-' and offer a-bc and a-de as valid choices. Instead it produces an additional a- for every time I hit tab. 

Same for Python3.1:

$ python3.1
Python 3.1.2 (release31-maint, Sep 26 2010, 13:51:01) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
>>> 
>>> def complete(text, state):
...   if (state == 0):
...     return "a-bc"
...   if (state == 1):
...     return "a-de"
...   else:
...     return None
... 
>>> 
>>> readline.parse_and_bind("Tab: complete")
>>> readline.set_completer(complete)
>>> 
>>> input()
a-a-a-a-a-
'a-a-a-a-a-'
>>> 


Other programming languages falling back to the GNU C-readline library don't have this problem. Consider the roughly equivalent example in Perl which works as expected:

use Term::ReadLine;

sub complete
{
        my ($text, $state) = @_;
        if ($state == 0)
        {
                return "a-bc";
        }
        elsif ($state == 1)
        {
                return "a-cd";
        }
        else
        {
                return undef;
        }
}

my $term = new Term::ReadLine 'sample';
my $attribs = $term->Attribs;

$term->parse_and_bind("Tab: complete");
$attribs->{completion_entry_function} = \&complete;

$term->readline();



Running it:

$ perl rl 
a-
a-bc  a-cd  
a-
msg156739 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-25 07:20
I can’t reproduce with 2.7 or 3.2 with a readline module linked to GNU readline.
msg174683 - (view) Author: Hideaki Takahashi (hideaki_t) * Date: 2012-11-03 20:08
I think this is the default behavior of readline module.

the default word delimiters for completion contains dash(-). so completion breaks at dash.

  >>> import readline
  >>> readline.get_completer_delims()
  ' \t\n`~!@#$%^&*()-=+[{]}\\|;:\'",<>/?'

In contrast, the default word delimitors of GNU readline is "\t\n\"\\'`@$><=;|&{(" and perl binding does not change it.

when I remove dash from delims like below, it works.

   >>> readline.set_completer_delims(readline.get_completer_delims().replace('-', ''))
   >>> input()
   a-
   a-bc  a-de
   a-
msg174686 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-11-03 20:17
I think I get it: Python sets custom delimiters that include '-' because for the Python REPL, it’s not possible to have one identifier with a dash, so it’s considered a delimiter and when you press tab, a new completion is started.  For a custom REPL however, you may want to have dashes as legal parts of your completed words, so you want to call set_completer_delims.  Reclassifying as a doc patch.
msg255316 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-25 02:24
I propose to address this with the general documentation bug, Issue 6953
msg262898 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-04-05 10:31
New changeset 6137c46cb8df by Martin Panter in branch '2.7':
Issue #6953: Rearrange and expand Readline module documentation
https://hg.python.org/cpython/rev/6137c46cb8df

New changeset b1acd6cf15b6 by Martin Panter in branch '3.5':
Issue #6953: Rearrange and expand Readline module documentation
https://hg.python.org/cpython/rev/b1acd6cf15b6
msg262925 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-04-05 22:23
My update includes a new section called Completion, with the following text:

By default, Readline is set up to be used by “rlcompleter” to complete Python identifiers for the interactive interpreter. If the “readline” module is to be used with a custom completer, a different set of word delimiters should be set.

I hope that is enough to call this fixed :)
History
Date User Action Args
2016-04-05 22:23:53martin.pantersetstatus: open -> closed
versions: + Python 3.5, Python 3.6, - Python 3.2, Python 3.3, Python 3.4
messages: + msg262925

resolution: fixed
stage: needs patch -> resolved
2016-04-05 10:31:45python-devsetnosy: + python-dev
messages: + msg262898
2015-11-25 02:24:57martin.pantersettitle: readline completion flaw -> Improve doc for readline.set_completer_delims()
nosy: + martin.panter

messages: + msg255316

dependencies: + readline documentation needs work
2012-11-03 20:17:08eric.araujosetassignee: docs@python
components: + Documentation, - Library (Lib)
versions: + Python 2.7, Python 3.2, Python 3.3, Python 3.4
keywords: + easy
nosy: + docs@python

messages: + msg174686
stage: needs patch
2012-11-03 20:08:27hideaki_tsetnosy: + hideaki_t
messages: + msg174683
2012-03-25 07:20:08eric.araujosetnosy: + eric.araujo

messages: + msg156739
versions: - Python 2.6, Python 3.1
2010-12-30 13:08:07rheisecreate