classification
Title: rlcompleter should be enabled automatically
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: belopolsky, cben, dabeaz, eric.araujo, ezio.melotti, flox, giampaolo.rodola, jason.coombs, larry, lesmana, loewis, mark.dickinson, ned.deily, pitrou, python-dev, r.david.murray, ronaldoussoren, serhiy.storchaka, steven.daprano, techtonik, tshepang, xdegaye
Priority: deferred blocker Keywords: patch

Created on 2009-04-25 23:27 by cben, last changed 2014-04-12 16:09 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
fix-5845.diff eric.araujo, 2011-08-18 17:05 review
c43e264256e4.diff pitrou, 2013-05-01 19:35 review
defaultreadline.patch pitrou, 2013-05-01 20:31 review
issue5845_osx_fix.patch mark.dickinson, 2013-05-06 12:34 review
completer.py steven.daprano, 2013-05-06 12:46
completer.py steven.daprano, 2013-07-31 08:04
history.py steven.daprano, 2013-07-31 08:04
rl_history_guard.patch pitrou, 2013-09-29 20:06 review
Repositories containing patches
http://hg.python.org/sandbox/merwok#fix5845
Messages (64)
msg86570 - (view) Author: Cherniavsky Beni (cben) * Date: 2009-04-25 23:27
An interactive prompt should offer working completion out-of-the-box,
without requiring every Python user on earth to create a $PYTHONSTARTUP
with '''import readline; readline.parse_and_bind("tab: complete")'''.

Note that it should work not only when Python is run without arguments,
but also when running with the -i option.
And ideally, it should only install the completion function when
dropping to the interpreter, not from the start, so that code that runs
before dropping to the interpreter is not affected.  But this is really
not important.

Safety: if 'rlcompleter' or 'readline' are not availiable (e.g. broken
python install), the user must still get a working interpreter, just
without completion.

Portability: there won't be completion on windows (as long as pyreadline
or similar is not part of Python).  But that shouldn't delay unix support!
msg86632 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2009-04-27 00:38
What would be the right place for this to happen? I first thought of
site.py, but -S would then turn off readline support. Does it have to be
done somewhere in C land?

But perhaps tab isn’t the right key to bind. I think inputrc could set
it to something different, perhaps shell rc files too. Is there an API
to get this setting, does readline.so already do the right thing, or
does it have to be added?

Moreover, in my PYTHONSTARTUP I not only bind tab to complete, but also
set a history file. Should this be automatic too? UNIX rules would
suggest a standard ~/.pyhistory file, or we could follow newer
freedesktop.org specifications [1].

So, I don’t know the history and choices behind readline.so, but from a
user point of view, being fed up with lack of tab completion finally had
me put up a PYTHONSTARTUP file with these settings, and then some useful
functions, and so on. This lack of completion was ultimately a good
thing for me. Still, completion could be made to work out of the box,
and the man page could link to the doc explaining how to customize this
behavior.

[1] XDG Base Directory Specification:
http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html

Cheers, Merwok
msg86660 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-04-27 12:48
He, adding completion is also something I find myself adding on every
box on which I use Python...
msg123696 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-12-09 20:49
for what it is worth, I am +1 on having completion and history file work by default.  The sqlite3 command line does this, for example.  I think it is what unix user expect nowadays, and I think it is reasonable.

Looking at my home directory, it would appear that the de-facto standard history file name would be .python_history.  This is based on .mysql_history, .psql_history, and .sqlite_history, none of which I configured explicitly to support history saving.
msg123697 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-12-09 20:54
Okay, that’s one question answered.  Still to solve: How to bind the right key? (“But perhaps tab isn’t the right key to bind. I think inputrc could set it to something different, perhaps shell rc files too. Is there an API to get this setting, does readline.so already do the right thing, or does it have to be added?”)
msg123698 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-12-09 20:58
I think TAB is the key expected by most people, so let's make it the default.
As for the location, site.py is an adequate one IMO.
msg123703 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2010-12-09 21:48
Keep in mind that the Python readline module may be linked to either GNU readline or the BSD editline (libedit) library and they have different command strings.  Note the warning here:

http://docs.python.org/dev/py3k/library/readline.html

Here's a snippet of what I have today in my startup file:

    import rlcompleter
    if 'libedit' in readline.__doc__:
        readline.parse_and_bind("bind ^I rl_complete")
    else:
        readline.parse_and_bind("tab: complete")

See Issue10666 for more details and some possible changes.
msg123705 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2010-12-09 22:30
Nosying Martin: any Windows installer concerns?
msg123709 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-12-09 23:12
There is no readline support on Windows at all, so I don't think the Windows installer can be affected. I'm uncertain what the proposed change to Python is at this point, though.
msg132117 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-03-25 17:08
Ned, does readline.read_init_file() with libedit?
msg132120 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-03-25 17:39
readline.read_init_file() does work with libedit.  The directives read have to be in libedit format.
msg132158 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-03-25 21:28
Please ignore the generated patches, left here for Martin’s benefit, and review the manually attached fix-5845.diff.

I have used read_init_file instead of hard-coding tab (it’s anyway the default for readline).

I may have repeated the same info in too much doc files, but I preferred to have more than one place where people could learn about the new behavior.  Advice welcomed.

If the basic approach looks good, especially if site still feels the right place for this, I will add tests.
msg142371 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-08-18 17:05
Updated patch (not using Mercurial, looks like I haven’t enough bandwidth to push all those changesets).
msg142393 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-08-18 18:32
> Updated patch (not using Mercurial, looks like I haven’t enough
> bandwidth to push all those changesets).

I think there's a bug in the way you are detecting interactive mode:

$ ./python -c "import sys; print(sys.stdin.isatty())"
True
msg142395 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-08-18 18:37
Interestingly, there's already the following code in Modules/main.c:

    if ((Py_InspectFlag || (command == NULL && filename == NULL && module == NULL)) &&
        isatty(fileno(stdin))) {
        PyObject *v;
        v = PyImport_ImportModule("readline");
        if (v == NULL)
            PyErr_Clear();
        else
            Py_DECREF(v);
    }


...meaning readline already gets imported automatically when desired. And indeed:

$ ./python -S
Python 3.3.0a0 (default:50f1922bc1d5, Aug 18 2011, 00:09:47) 
[GCC 4.5.2] on linux2
>>> import sys
>>> sys.modules['readline']
<module 'readline' from '/home/antoine/cpython/default/build/lib.linux-x86_64-3.3-pydebug/readline.cpython-33dm.so'>


So perhaps we could simply change this code to import another, private module (e.g. "_setupinteractive.py") which would setup readline and rlcompleter?
msg143548 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-09-05 16:30
We can’t just decide to enable completion by checking “'readline' in sys.modules” in site, because it always returns False.  site is imported in Python/pythonrun.c, and I guess that Modules/main.c is run afterwards.  More importantly, I think the import of readline in Modules/main.c is a CPython implementation detail, and I’d prefer to have code in site that is directly useful for other VMs.  (I’d go as far as proposing to remove the import readline from main.c, when my patch makes it obsolete.)

I re-read the docs for sys.argv and the -m switch and decided to translate to C checks (“command == NULL && filename == NULL && module == NULL”) to “if not sys.argv[0]”.  Unfortunately, sys.argv is not available when site is imported, so this is a dead end :(

So, I could try your _setupinteractive module idea, even though I think it’s a bit unclean, or drink the cool aid and set up completion in main.c.
msg143554 - (view) Author: Cherniavsky Beni (cben) * Date: 2011-09-05 18:29
Easily detecting interactive mode is of general interest for customization.

1. What if C also set sys.flags.interactive in "python" mode,
   or exposed sys.flags.implicit_interactive (but with better name)?

2. It's more useful to have a hook called when entering interactive mode,
   rather than a flag that's set from the beginning:

   $ python -i -c 'import sys; print sys.flags.interactive'
   1
   >>> 

   For this, importing _setupinteractive is a step forward;
   calling e.g. sys.__interactivehook__ sounds even better.
   (site.py would set it by default to a function that enables
   rlcompleter, user can always override...)

BTW, drawback of doing any such setup in site.py: "python -S" would 
be unfriendly!
msg143608 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-09-06 14:51
> Easily detecting interactive mode is of general interest for
> customization.
Thanks for the feedback.  We could open a feature request for that, and/or ask python-ideas.

> What if C also set sys.flags.interactive in "python" mode, or exposed
> sys.flags.implicit_interactive (but with better name)?
An attribute in the sys module sounds like a good idea.  It would not be an attribute of sys.flags though, as those are strictly command-line arguments.

> It's more useful to have a hook called when entering interactive mode, rather than a flag
> that's set from the beginning:
We already have such a hook: $PYTHONSTARTUP

> calling e.g. sys.__interactivehook__ sounds even better.
That’s another interesting idea.

> BTW, drawback of doing any such setup in site.py: "python -S" would be unfriendly!
People using -S don’t want the customizations done in site, so I don’t think there’s a problem here.
msg143609 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-09-06 14:54
> > It's more useful to have a hook called when entering interactive mode, rather than a flag
> > that's set from the beginning:
> We already have such a hook: $PYTHONSTARTUP

$PYTHONSTARTUP doesn't work with -i
msg143618 - (view) Author: Cherniavsky Beni (cben) * Date: 2011-09-06 16:05
[sorry, html mail was bad idea]

On Tue, Sep 6, 2011 at 17:54, Antoine Pitrou <report@bugs.python.org> wrote:

Éric Araujo <merwok@netwok.org> added the comment:
> > It's more useful to have a hook called when entering interactive mode,
> > rather than a flag
> > that's set from the beginning:
> We already have such a hook: $PYTHONSTARTUP

Good point!
It covers the user's desire customization very well (esp. if it worked with -i).
sys.__interactivehook__ has the benefit of being cleanly settable from python code.
But it might well be a YAGNI idea.

> $PYTHONSTARTUP doesn't work with -i
 
Perhaps it should?
I can't think of a thing that makes sense in $PYTHONSTARTUP that I wouldn't want with -i.
(and if there is one, one can add a test for sys.flags.interactive, or run with env PYTHONSTARTUP='')
 
Point to watch out for: errors in $PYTHONSTARTUP.
One of the uses of "python -i script.py" is doing pdb.pm() on an exception thrown by the script;
ideally a broken $PYTHONSTARTUP would not overr

> > BTW, drawback of doing any such setup in site.py: "python -S" 
> > would be unfriendly!
> People using -S don’t want the customizations done in site, so I 
> don’t think there’s a problem here.

python -S doesn't disable readline.  What makes completions more of a "customization" than editing?
The fact that it'd be implemented in site.py?
Yes, obviously, if it's implemented in site.py, -S should disable it.  My point was that it doesn't have to be implemented there.  You could drink the cool aid instead :-)
msg143621 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-09-06 16:14
> sys.__interactivehook__ has the benefit of being cleanly settable from python code.
> But it might well be a YAGNI idea.
I’ll ask python-dev about that.  For the moment, I prefer the idea of a new sys.interactive attribute (boolean).

>> $PYTHONSTARTUP doesn't work with -i
> Perhaps it should?
-i means two things: interactive and inspect.  The second meaning is used when you want python to execute a script or module just as usual but without exiting afterward, to let you inspect the state of the program.  Supporting PYTHONSTARTUP would change these use cases and be an unacceptable change.

 
>> BTW, drawback of doing any such setup in site.py: "python -S" 
>> would be unfriendly!
> People using -S don’t want the customizations done in site, so I 
> don’t think there’s a problem here.

> python -S doesn't disable readline.
> What makes completions more of a "customization" than editing?
> The fact that it'd be implemented in site.py?
If you go back to the first messages of this report, you’ll see that I wasn’t sure what a good location would be, and decided site because it did not look out of place.  There isn’t a stronger rationale than that.  One could argue that even now, site mixes disparate functionality: one program could wish to use -S to disable sys.path munging but run into bugs because the builtin exit is not defined anymore for example.

> Yes, obviously, if it's implemented in site.py, -S should disable it.
We agree.

> My point was that it doesn't have to be implemented there.
I really want to explore all options before renouncing to implement it in site.  I’d like the implementation to be maintainable and shareable with other Python VMs.
msg143961 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-09-13 12:39
FTR, I tried checking sys.ps1 instead of argv but it’s the same problem.
msg188241 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-01 20:31
In the spirit of pushing this forward, here is an updated patch using the sys.__interactivehook__ approach. I didn't add any tests since it doesn't seem very easy to write any.

If nobody objects, I would like to commit this soon.
msg188385 - (view) Author: Roundup Robot (python-dev) Date: 2013-05-04 18:08
New changeset d5ef330bac50 by Antoine Pitrou in branch 'default':
Issue #5845: Enable tab-completion in the interactive interpreter by default, thanks to a new sys.__interactivehook__.
http://hg.python.org/cpython/rev/d5ef330bac50
msg188386 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-04 18:11
End-users will hopefully rejoice :)
msg188505 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2013-05-06 11:06
On OS X 10.6, I now get the following message at interpreter startup:


iwasawa:cpython mdickinson$ ./python.exe
Python 3.4.0a0 (default:d5ef330bac50, May  6 2013, 13:05:57) 
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Failed calling sys.__interactivehook__
Traceback (most recent call last):
  File "/Users/mdickinson/Python/cpython/Lib/site.py", line 481, in register_readline
    readline.read_init_file()
OSError: [Errno -1] Unknown error: -1
msg188511 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2013-05-06 11:21
More information: readline.read_init_file() produced the same traceback before this commit, so all that's changed is that we're now calling this at interpreter startup.  It looks like I'm using the system libedit:

iwasawa:cpython mdickinson$ otool -L build/lib.macosx-10.6-x86_64-3.4-pydebug/readline.so 
build/lib.macosx-10.6-x86_64-3.4-pydebug/readline.so:
	/usr/lib/libedit.2.dylib (compatibility version 2.0.0, current version 2.11.0)
	/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.11)
iwasawa:cpython mdickinson$
msg188517 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2013-05-06 12:20
So I'm failing to find any documentation for libedit, but it looks as though this error occurs if rl_read_init_file fails to find an .editrc file in the appropriate place.  If I create an empty .editrc file in my home directory, the error disappears.  (Having an .inputrc file doesn't seem to make a difference either way.)

Perhaps the solution is to ignore an OSError from readline.read_init_file in register_readline---i.e., add a try / except OSError there.  The other option would be to ignore a nonzero errno from rl_read_init_file, but that doesn't sound so good: I'd expect someone deliberately calling readline.read_init_file to get an exception if the initialization file isn't found.
msg188521 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-06 12:27
> So I'm failing to find any documentation for libedit, but it looks as
> though this error occurs if rl_read_init_file fails to find an
> .editrc file in the appropriate place.  If I create an empty .editrc
> file in my home directory, the error disappears.  (Having an
> .inputrc file doesn't seem to make a difference either way.)
> 
> Perhaps the solution is to ignore an OSError from
> readline.read_init_file in register_readline---i.e., add a try /
> except OSError there.

This sounds fine to me. Can you propose a patch? I'm unlikely to ever have
an OS X machine.
msg188523 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2013-05-06 12:34
The attached fixes the issue for me.  I'm not sure whether the try / except should only be done on Apple, though.  What's the behaviour on Linux if there's no .inputrc file?
msg188524 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-06 12:42
> The attached fixes the issue for me.  I'm not sure whether the try /
> except should only be done on Apple, though.  What's the behaviour
> on Linux if there's no .inputrc file?

Everything works fine under Linux (as usual :-)).
There's no need to restrict the try / except to Apple platforms, though.
msg188525 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2013-05-06 12:46
I'm not able to test the patch at the moment, but since it essentially just uses the recipe in the docs, I expect it will have the same side-effect. Namely, it prevents you using the tab key to indent in the interactive interpreter.

Now I don't know if I'm missing something painfully obvious, but having to bang out space-space-space-space for every indent is surely not going to be a win for usability ;-) Even if I am missing something, surely so will a lot of other users.

I use a readline completer that (in my opinion) does the Right Thing: at the start of the line, hitting TAB inserts a tab, otherwise it does completion. It also sets a history file, adds a couple of key bindings that I sometimes find useful, and wraps it all up in a class. (Attached.) If you think there is any value in this, I'm happy to update it to Python 3.3 and polish it up as a patch to the readline module. Or someone can just mine it for ideas.
msg188532 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-06 13:24
> Now I don't know if I'm missing something painfully obvious, but
> having to bang out space-space-space-space for every indent is
> surely not going to be a win for usability ;-) Even if I am missing
> something, surely so will a lot of other users.

What *looks* painfully obvious to me is that Python doesn't force you
to use 4 spaces (or tabs) for indents :-)

> I use a readline completer that (in my opinion) does the Right Thing:
> at the start of the line, hitting TAB inserts a tab, otherwise it
> does completion. It also sets a history file, adds a couple of key
> bindings that I sometimes find useful, and wraps it all up in a
> class.

The "indent at beginning of line" thing sounds useful. The rest seems
to conflate the concept of a completer with the readline module itself.
Even though it's called rlcompleter, the rlcompleter module is
actually generic and could be used with other input schemes, so I don't
think it's the right place to set history options or key bindings.

In any case, you should post your patch as part of a separate issue, IMO.
msg188533 - (view) Author: Roundup Robot (python-dev) Date: 2013-05-06 13:39
New changeset 82e92da929eb by Mark Dickinson in branch 'default':
Issue #5845: avoid an exception at startup on OS X if no .editrc file exists.
http://hg.python.org/cpython/rev/82e92da929eb
msg188534 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2013-05-06 13:40
Applied the OS X fix;  reclosing.
msg188535 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-05-06 13:44
The tab-doesn't-indent still needs to be fixed before 3.4 is released.
msg188540 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-06 14:06
> The tab-doesn't-indent still needs to be fixed before 3.4 is
> released.

I don't really understand why this would be a release blocker?
The interpreter prompt is a convenience, we don't guarantee
compatibility like with stdlib APIs.
msg188542 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-05-06 14:14
It is a release blocker because it is a major usability regression.
msg188543 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-05-06 14:17
Just to be clear how important I consider this, I would advocate for backing out this patch rather than releasing 3.4 with a broken tab key at the interactive prompt.  But I'd rather have the patch *and* a working tab key.
msg188549 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-06 14:40
> It is a release blocker because it is a major usability regression.

Really? You can press the space key to indent, it works as well
as the tab key...
msg188552 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-05-06 14:44
Well, post it to python-dev and see what reaction you get :)

I could be wrong, but I don't think I am (obviously).
msg188553 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2013-05-06 14:45
I expect that a lot of users use the tab key to indent in the repl (as well as in editors, smart enough editors can convert the tab presses to spaces)
msg188555 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-06 14:52
> Well, post it to python-dev and see what reaction you get :)

I'm not interested in python-dev reactions here. For this
kind of issue, we'll have hundreds of messages for and against
the change without any useful content.

The commit adds an often-requested feature (to the point that many
people were already taking the pain of activating it manually).
But, yes, we could add Steven's refinement so that all bases are
covered.
msg188557 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-06 14:57
> I expect that a lot of users use the tab key to indent in the repl
> (as well as in editors, smart enough editors can convert the tab
> presses to spaces)

The interpreter prompt is not a text editor at all. You can misuse
it as one, but that's a loss of time and energy IMO.

Really, everyone I've ever showed tab-completion to has always
been enthusiastic about it ("why isn't this enabled by default?").
None has ever raised the concern of being able to use the tab key
to indent code under the prompt.

(probably because noone is crazy enough to type long chunks of
code under the interactive prompt anyway)
msg188560 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2013-05-06 15:34
I am in AP's camp on the tab issue, but I think we can preserve "tab inserts tab" behavior at the continuation prompt.  I don't like "indent at beginning of line."  I have rlcompleter enabled in Python 2.6 and i get the following when I press tab:

Python 2.6.6 (r266:84292, Feb 22 2013, 00:00:18)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> <tab>
Display all 173 possibilities? (y or n) y
ArithmeticError(            abs(                        input(
AssertionError(             all(                        int(
AttributeError(             and                         intern(
BaseException(              any(                        is
..

I find this rather useful.

At the ... prompt, however, a tab (or better four spaces) is arguably the right completion at the beginning of the line.
msg188589 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-06 20:36
Take it with a large grain of Internet salt, but Ezio pointed me to the following reactions:
http://www.reddit.com/r/Python/comments/1dq7sh/python_repl_finally_gets_tab_completion_by_default/
msg188592 - (view) Author: Tshepang Lekhonkhobe (tshepang) * Date: 2013-05-06 20:55
Also made it to the front-page of Hacker News[1], with better quality comments than the reddit thread.

[1] https://news.ycombinator.com/item?id=5658062
msg193981 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-07-31 07:14
This issue seems to have stalled.  But it's still marked as a release blocker, which means I can't release Python 3.4a1 in two days if this issue is still open.

One of three things will happen:
1) This issue is marked "closed".
2) This issue is downgraded from "release blocker".
3) The alpha slips.

IMO the optimal solution is that tab preceded by only whitespace indents, and tab preceded by any non-whitespace character attempts to complete.  Can we goad readline into behaving this way?
msg193986 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2013-07-31 08:04
On 31/07/13 17:14, Larry Hastings wrote:

> IMO the optimal solution is that tab preceded by only whitespace indents, and tab preceded by any non-whitespace character attempts to complete.  Can we goad readline into behaving this way?

Yes we can. Attached are a pair of lightweight modules I've used for this for the last 3-4 years, and it works fine in Python 2.4 through 3.3. I put something like this in my startup.py:

import completer
import history
history = history.History()
completer = completer.Completer(
         bindings=(r'"\C-xo": overwrite-mode',
                   r'"\C-xd": dump-functions',
                  ))

Originally they were in a single class, but I was persuaded that it was cleaner to separate them.
msg194006 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2013-07-31 18:03
If R. David agrees (assuming no patch is coming forward), it could be degrated to deferred-blocker, and the alpha phases could be used to see how annoyed people are with "tab not working".
msg194010 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-07-31 19:42
Yeah, deferred blocker is fine with me.
msg194427 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-08-04 21:15
Do you know that the readline module first force rebind of TAB to insert-tab?
msg198636 - (view) Author: Xavier de Gaye (xdegaye) * Date: 2013-09-29 19:41
There is a backward compatibility issue with changeset d5ef330bac50
that enables tab-completion in the interactive interpreter by default.

When a user is not aware of this new feature and has been implementing
up to now her/his PYTHONSTARTUP file with the first example given in
python 3.3 documentation at

http://docs.python.org/3/library/readline.html?highlight=readline#example

then each new interactive python 3.4 session reads (appending them)
both history files which are saved on exit. Thus those files double
their size after each interactive python 3.4 session and pretty soon
become few mega bytes long with a python start up time that becomes
quite noticeable.
msg198640 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-09-29 20:00
> When a user is not aware of this new feature and has been implementing
> up to now her/his PYTHONSTARTUP file with the first example given in
> python 3.3 documentation at
> 
> http://docs.python.org/3/library/readline.html?highlight=readline#example
> 
> then each new interactive python 3.4 session reads (appending them)
> both history files which are saved on exit. Thus those files double
> their size after each interactive python 3.4 session and pretty soon
> become few mega bytes long with a python start up time that becomes
> quite noticeable.

Interesting. A way to avoid doing this would be to only load history if
no history is already present. Since the sys.__interactivehook__ is
executed after PYTHONSTARTUP, this would allow the latter to take
precedence.
msg198642 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-09-29 20:06
Attaching patch to only load history file if no history exists.
msg198645 - (view) Author: Roundup Robot (python-dev) Date: 2013-09-29 20:19
New changeset 687dd81cee3b by Antoine Pitrou in branch 'default':
Issue #5845: In site.py, only load readline history from ~/.python_history if no history has been read already.  This avoids double writes to the history file at shutdown.
http://hg.python.org/cpython/rev/687dd81cee3b
msg198648 - (view) Author: Xavier de Gaye (xdegaye) * Date: 2013-09-29 20:43
The patch fixes the problem on my setup. A very minor glitch: after
manually emptying or removing the PYTHONSTARTUP history file, the
history is loaded with the content of ~/.python_history on the next
session.
msg198653 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-09-29 21:13
> The patch fixes the problem on my setup. A very minor glitch: after
> manually emptying or removing the PYTHONSTARTUP history file, the
> history is loaded with the content of ~/.python_history on the next
> session.

Yes, the only way to know if a history file has already been loaded with
readline seems to be to query the history itself. So, if the history is
empty, we consider no file has been loaded.
msg205569 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-12-08 15:02
What is the status of this issue?
msg205595 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-12-08 19:18
I would say it's now closed. If there's some fine tuning needed, separate issues should be opened.
msg209472 - (view) Author: Jason R. Coombs (jason.coombs) * (Python committer) Date: 2014-01-27 20:13
Added issue20411, revealed in 3.4b2.
msg212972 - (view) Author: Roundup Robot (python-dev) Date: 2014-03-09 19:17
New changeset 69c451851c71 by R David Murray in branch 'default':
whatsnew: sys.__interactivehook__. (#5845)
http://hg.python.org/cpython/rev/69c451851c71
msg215784 - (view) Author: David Beazley (dabeaz) Date: 2014-04-09 00:17
Funny thing, this feature breaks the interactive interpreter in the most basic way on OS X systems.   For example, the tab key won't even work to indent.  You can't even type the most basic programs into the interactive interpreter. For example:

>>> for i in range(10):
...      print(i)

Oh sure, you can make it work by typing the space bar a bunch of times, but it's extremely annoying.  

The only way I was able to get a working interactive interpreter on my machine was to manually edit site.py and remove the call to enablerlcompleter() from main().

I hope someone reconsiders this feature and removes it as default behavior.
msg215975 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-04-12 16:09
This issue should have gone back to being a release blocker after the alpha release to fix the tab-as-indent issue, but obviously that didn't happen (I forgot about it myself).  Please open a new issue requesting a fix for this bug (that tab doesn't work as indent at the ... prompt), referencing the discussion in this issue.
History
Date User Action Args
2014-04-12 16:09:43r.david.murraysetmessages: + msg215975
2014-04-09 00:17:44dabeazsetnosy: + dabeaz
messages: + msg215784
2014-03-09 19:17:41python-devsetmessages: + msg212972
2014-01-27 20:13:40jason.coombssetnosy: + jason.coombs
messages: + msg209472
2013-12-08 19:18:07pitrousetstatus: open -> closed

messages: + msg205595
2013-12-08 15:02:34serhiy.storchakasetmessages: + msg205569
2013-09-29 21:13:22pitrousetmessages: + msg198653
2013-09-29 20:43:44xdegayesetmessages: + msg198648
2013-09-29 20:19:15python-devsetmessages: + msg198645
2013-09-29 20:06:57pitrousetkeywords: - needs review
resolution: fixed
messages: + msg198642

files: + rl_history_guard.patch
2013-09-29 20:00:50pitrousetmessages: + msg198640
2013-09-29 19:41:03xdegayesetnosy: + xdegaye
messages: + msg198636
2013-08-04 21:15:42serhiy.storchakasetmessages: + msg194427
2013-07-31 19:42:20r.david.murraysetpriority: release blocker -> deferred blocker

messages: + msg194010
2013-07-31 18:03:41loewissetmessages: + msg194006
2013-07-31 08:04:14steven.dapranosetfiles: + completer.py, history.py

messages: + msg193986
2013-07-31 07:14:13larrysetmessages: + msg193981
2013-06-26 10:36:36techtoniksetnosy: + techtonik
2013-05-06 20:55:20tshepangsetmessages: + msg188592
2013-05-06 20:36:36pitrousetmessages: + msg188589
2013-05-06 15:34:45belopolskysetmessages: + msg188560
2013-05-06 14:57:07pitrousetmessages: + msg188557
2013-05-06 14:52:07pitrousetmessages: + msg188555
2013-05-06 14:45:27ronaldoussorensetmessages: + msg188553
2013-05-06 14:44:02r.david.murraysetmessages: + msg188552
2013-05-06 14:40:06pitrousetmessages: + msg188549
2013-05-06 14:17:58r.david.murraysetmessages: + msg188543
2013-05-06 14:14:42r.david.murraysetmessages: + msg188542
2013-05-06 14:06:52pitrousetmessages: + msg188540
2013-05-06 13:44:31r.david.murraysetstatus: closed -> open
priority: normal -> release blocker

nosy: + larry
messages: + msg188535

resolution: fixed -> (no value)
2013-05-06 13:40:47mark.dickinsonsetstatus: open -> closed
resolution: fixed
messages: + msg188534
2013-05-06 13:39:47python-devsetmessages: + msg188533
2013-05-06 13:24:02pitrousetmessages: + msg188532
2013-05-06 12:46:53steven.dapranosetfiles: + completer.py

messages: + msg188525
2013-05-06 12:42:32pitrousetmessages: + msg188524
2013-05-06 12:34:14mark.dickinsonsetfiles: + issue5845_osx_fix.patch

messages: + msg188523
2013-05-06 12:27:25pitrousetmessages: + msg188521
2013-05-06 12:20:34mark.dickinsonsetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg188517
2013-05-06 11:21:21mark.dickinsonsetmessages: + msg188511
2013-05-06 11:06:12mark.dickinsonsetnosy: + mark.dickinson
messages: + msg188505
2013-05-04 18:11:40pitrousetstatus: open -> closed
resolution: fixed
messages: + msg188386

stage: patch review -> resolved
2013-05-04 18:08:43python-devsetnosy: + python-dev
messages: + msg188385
2013-05-02 07:37:39serhiy.storchakasetnosy: + serhiy.storchaka
2013-05-01 20:31:41pitrousetfiles: + defaultreadline.patch

messages: + msg188241
2013-05-01 19:35:41pitrousetfiles: + c43e264256e4.diff
2013-03-21 23:51:05tshepangsetversions: + Python 3.4, - Python 3.3
2013-03-21 23:49:41tshepangsetnosy: + tshepang
2013-03-21 12:23:54floxsetnosy: + flox
2013-03-21 09:58:20giampaolo.rodolasetnosy: + giampaolo.rodola
2012-10-05 15:43:57steven.dapranosetnosy: + steven.daprano
2011-09-13 12:39:18eric.araujosetmessages: + msg143961
2011-09-08 10:20:25ezio.melottisetnosy: + ezio.melotti
2011-09-06 16:14:42eric.araujosetmessages: + msg143621
2011-09-06 16:06:06eric.araujosetmessages: - msg143617
2011-09-06 16:05:46eric.araujosetfiles: - unnamed
2011-09-06 16:05:05cbensetmessages: + msg143618
2011-09-06 15:54:26cbensetfiles: + unnamed

messages: + msg143617
2011-09-06 14:54:39pitrousetmessages: + msg143609
2011-09-06 14:51:23eric.araujosetmessages: + msg143608
2011-09-05 18:29:38cbensetmessages: + msg143554
2011-09-05 16:30:44eric.araujosetmessages: + msg143548
2011-08-18 18:37:53pitrousetmessages: + msg142395
2011-08-18 18:32:07pitrousetmessages: + msg142393
2011-08-18 17:05:24eric.araujosetfiles: + fix-5845.diff

messages: + msg142371
2011-08-18 17:04:40eric.araujosetfiles: - fix-5845.diff
2011-03-25 23:25:04eric.araujosetfiles: - d1cbf0347eb4.diff
2011-03-25 23:25:01eric.araujosetfiles: - c43e264256e4.diff
2011-03-25 23:24:56eric.araujosetfiles: - a5bded1b8db4.diff
2011-03-25 23:18:48eric.araujosetfiles: + d1cbf0347eb4.diff
2011-03-25 22:21:16loewissetfiles: + c43e264256e4.diff
2011-03-25 22:15:42loewissetfiles: - c43e264256e4.diff
2011-03-25 21:28:17eric.araujosetkeywords: + needs review
files: + fix-5845.diff
messages: + msg132158

stage: patch review
2011-03-25 21:16:17eric.araujosetfiles: + c43e264256e4.diff
2011-03-25 21:15:09eric.araujosetfiles: + a5bded1b8db4.diff
keywords: + patch
2011-03-25 20:26:05eric.araujosethgrepos: + hgrepo9
2011-03-25 17:39:41ned.deilysetmessages: + msg132120
2011-03-25 17:08:32eric.araujosetmessages: + msg132117
versions: - Python 3.2
2010-12-09 23:12:51loewissetmessages: + msg123709
2010-12-09 22:30:22ned.deilysetnosy: + loewis
messages: + msg123705
2010-12-09 21:48:17ned.deilysetmessages: - msg123702
2010-12-09 21:48:10ned.deilysetmessages: + msg123703
2010-12-09 21:47:21ned.deilysetnosy: + ronaldoussoren, ned.deily
messages: + msg123702
2010-12-09 20:58:58pitrousetnosy: cben, belopolsky, pitrou, eric.araujo, r.david.murray, lesmana
messages: + msg123698
components: + Library (Lib), - Interpreter Core
versions: + Python 3.2
2010-12-09 20:54:53eric.araujosetmessages: + msg123697
versions: + Python 3.3, - Python 3.1, Python 2.7
2010-12-09 20:49:07r.david.murraysetnosy: + r.david.murray
messages: + msg123696
2010-06-12 01:45:23belopolskysetnosy: + belopolsky
2010-06-06 15:32:04lesmanasetnosy: + lesmana
2009-04-27 12:48:49pitrousetnosy: + pitrou
messages: + msg86660
2009-04-27 00:38:22eric.araujosetnosy: + eric.araujo
messages: + msg86632
2009-04-25 23:28:27cbensettype: enhancement
components: + Interpreter Core
versions: + Python 3.1, Python 2.7
2009-04-25 23:27:18cbencreate