New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tabs don't work correctly in python repl #69846
Comments
When Python is compiled with readline, repeatedly pressing <TAB> key in repl breaks the repl prompt. Below is what I see when I hit <TAB> 4 times.
Reproducible when Python is built from source on latest OS X. Works OK when installed from MacPorts. Further, if I add PyOS_ReadlineFunctionPointer = PyOS_StdioReadline; in PyOS_Readline in myreadline.c, everything works as expected, so I think this is a readline bug. |
This was reported (on twitter) by David Beazly as well, as applying to the OSX downloaded from python.org. It works fine on linux for me. Is it readline-version dependent, or a bug in the readline shipped with OSX? |
Is this related to the BSD editline library (libedit), or is the actual Gnu Readline library being used? Apparently you can check for "libedit" in readline.__doc__ to tell the difference. |
Great suggestion. So on my machine, macports version (no bug) is built against readline; manual build from source -- libedit. So apparently, this is a libedit thing. |
David Beazley’s description of the bug: “hitting tab inserts spaces and a carriage return at the same time.” |
Applying my patch at bpo-13501, then building with “autoreconf && ./configure --with-readline=editline”, I can reproduce the funny tab behaviour on Linux. |
I think this might just be a side effect of the way we abuse the tab completer to insert a literal tab (bpo-23441, revision 82ccdf2df5ac). If I change the code to insert the letter T instead of tabs: if not text.strip('T'):
if state == 0:
return text + 'T'
else:
return None I see this behaviour:
Illustration: >>> TTTT <== Typed T three times, then Tab twice
TTTTT <== Completion list from second Tab press
>>> TTTTT
TTTTTT
>>> TTTTTT |
Attached is a patch that uses different logic for tabulation. Instead of returning '\t' from the completion callback, it instead calls explicit readline API: "readline.insert_text('\t'); readline.redisplay()" Please review. |
Can someone try the patch? I'm fairly confident it should work, and it'd be great to have it fixed in 3.5.1. |
Sorry to throw a potential spanner in the works, but I think this introduces a dependence on the readline module. Consider what happens when the Completer class is used but Readline is not, or even if the “readline” could not be imported (see bottom of rlcompleter.py). Maybe there is a way around this, but it might involve more complicated code. |
Didn’t mean to adjust priority |
Hm... This module provides a "Completer" class **specifically** for readline, as its documentation (and name!) indicates. |
The "release blocker" priority level is inappropriate for nice-to-have features. Quoting PEP-101, the how-to-make-a-Python-release guide:
While I'd like to see this fixed too, I'm not holding up the release for it. |
The thing is, when it is a regression it should be fixed before the release is issued, even if it is a more "minor" issue. Otherwise, especially in a x.y.1 release, we look pretty bad. However, this one had been broken in the field so long that rule doesn't really apply, I think :) At least it is better in 3.5 and only affects one platform now, instead of all of them. If people didn't have time to get it fixed and committed before your deadline, that's too bad, I'm afraid :(. David Beazly is going to be pissed, though... |
No, it affects any platform where CPython is compiled with libedit instead of readline. And even if it was one platform - OS X is big, bigger than Windows probably in terms of Python users.
I have a fix (attached to the issue). I think it's OK, but I need someone else to test it and review it. Martin, does the patch fix the problem on Linux with libedit? Do we use libedit on Windows? |
Regarding the purpose of the “Readline completer” module, I agree it is inconvenient. But the documentation <https://docs.python.org/3.5/library/rlcompleter.html\> does say “without readline, the Completer class . . . can still be used”. I ran into this problem in bpo-25419 where Serhiy pointed it out. OS X is apparently the main platform where Editline is supported. There are even #ifdef __APPLE__ bits to work around bugs. However, people also seem to use Editline on Free BSD. Testing your current patch on Linux + Editline, it is worse than the 3.5 behaviour. Pressing Tab once appears to work, until you type the next key. It seems to invoke the history search mode (normally done with Ctrl+R), except the “bck:” indicator is not immediately shown. So the next key brings up an old history line, rather than being inserted directly. But if this works fine on OS X’s Editline, then maybe my setup is not right. I also see annoying buffering or flushing problems with my Editline setup. I am using Arch Linux’s libedit-20150325_3.1-1 package. Testing with Linux + Gnu Readline, it seems to work as advertised. I haven’t heard of Editline or Gnu Readline being used on Windows, certainly not in normal setups. |
It seems to be the readline.redisplay() line that triggers the search mode. If I comment out that line, I am back to the 3.5 behaviour that adds extra lines. |
We try our best to support libedit, but support for it did come after readline, so there are issues with the support. However, if we are (as we are) enabling completion by default, I think that has to work with libedit, since that's what is used on OSX which, as Yury said, is a major platform for Python. IIRC, on windows one has to install something 3rd party (pyreadline?) to get readline support. |
Yury or anyone else: can you confirm if the patch works properly with a proper OS X or BSD libedit (rather than my questionable Linux version?). If so, I think it is okay to ignore my Linux problems, and I could look at fixing the compatibility when the readline module is not available. Another way to look at this may be to fix the Editline (libedit) library to more faithfully emulate the Gnu version. |
With rlcompleter.patch applied, I see the same behavior on my OS X (with libedit) and Ubuntu (with readline) systems. Thanks! |
LGTM on OS X with both the system libedit and a MacPorts GNU readline. Let's apply this. |
New changeset 64417e7a1760 by Yury Selivanov in branch '3.5': New changeset 87dfadd61e0d by Yury Selivanov in branch 'default': |
Merged. Martin, Berker, and Ned, thanks for testing this patch out. |
Thanks, Yury! Look like we forgot to update test_rcompleter: ====================================================================== Traceback (most recent call last):
File "/ssd/buildbot/buildarea/3.5.gps-ubuntu-exynos5-armv7l/build/Lib/test/test_rlcompleter.py", line 82, in test_complete
self.assertEqual(completer.complete('', 0), '\t')
AssertionError: '' != '\t'
+ http://buildbot.python.org/all/builders/ARMv7%20Ubuntu%203.5/builds/576/steps/test/logs/stdio |
See also the Windows buildbots, where the Readline module is not available, but “rlcompleter” still (used to) work. |
Windows issue is a blocker so I suggest reverting 64417e7a1760. We could apply the same fix in readline.c for 3.5.x but it would probably be a bit hackish. |
Berker, I'll fix the windows issue in a few hours. Sorry for breaking things. |
New changeset 980ea968444c by Yury Selivanov in branch '3.5': |
Everything should be OK now (both broken tests and using rlcompleter on Windows). Please confirm. |
Could this commit be the reason why the attached code behaves differently in 2.7 and 3.5.2? This is the code used by Emacs' default Python mode to do completion; with it (python -i completion.py), pressing "tab" on a plain prompt offers candidates in Python 2.7, but it doesn't in Python 3.5.2 (instead, it inserts a tab). |
Thanks Martin for the clarification! I'll leave it to you to decide whether there is something to fix here (I'll admit to not understanding much of that code). |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: