Title: interactive mode TAB does not insert on OS X built with editline instead of GNU readline
Type: behavior Stage: resolved
Components: macOS Versions: Python 3.2, Python 2.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ronaldoussoren Nosy List: benjamin.peterson, georg.brandl, ned.deily, r.david.murray, ronaldoussoren, zvezdan
Priority: release blocker Keywords: patch

Created on 2010-09-20 21:36 by ned.deily, last changed 2010-12-18 03:52 by r.david.murray. This issue is now closed.

File name Uploaded Description Edit
issue9907-py3k.patch ned.deily, 2010-09-21 09:19
issue9907-27.patch ned.deily, 2010-09-21 09:20
issue9907-py3k-ronald.patch ronaldoussoren, 2010-09-28 15:15 review
Messages (8)
msg116981 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2010-09-20 21:36
When building Python on OS X, there is now support for linking Python with the readline compatibility interface of the OS X supplied BSD editline library rather than using the GNU readline library.  Because of deficiencies in the version in earlier OS X releases, this support is only enabled for builds with a deployment target of 10.5 or higher.  With the python 2.7 release, for the first time a installer for OS X is available that uses this capability: the 10.5 and higher 32-bit/64-bit version. The 10.3 and higher 32-bit-only installer uses GNU readline as do previous installers.  There is a behavior regression in the editline-linked versions: when started in interactive mode, the TAB key does not insert, rather it inserts a "./" file spec in the command buffer and a second TAB causes a completion search of files in the current directory.

With readline and typing <TAB> <CR>:

  $ python2.7
  Python 2.7 (r27:82508, Jul  3 2010, 20:17:05) 
  [GCC 4.0.1 (Apple Inc. build 5493)] on darwin
  Type "help", "copyright", "credits" or "license" for more information.

With editline and <TAB> <CR>:

  $ python2.7
  Python 2.7 (r27:82508, Jul  3 2010, 21:12:11) 
  [GCC 4.0.1 (Apple Inc. build 5493)] on darwin
  Type "help", "copyright", "credits" or "license" for more information.
  >>> ./
    File "<stdin>", line 1
  SyntaxError: invalid syntax
  >>> ^D

Two workarounds for python2.7 until the problem is addressed in a future installer:

   (1) either install the 10.3 and up python 2.7

or (2) add or edit a python startup file for python2.7:

       $ cat > $HOME/.pystartup
       import readline
       if 'libedit' in readline.__doc__:
           readline.parse_and_bind("bind ^I ed-insert")
       $ export PYTHONSTARTUP=$HOME/.pystartup

Since whitespace is significant in Python, Modules/readline.c initialization attempts to override TAB behavior by forcing TAB to "insert" by default (unless overridden by later readline module calls).  Somehow that is failing when going through editline's readline compatibility layer.
msg116986 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2010-09-20 21:58
[Thanks to Nik Krumm for reporting the problem on python-list/comp.lang.python]
msg117042 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2010-09-21 09:19
The problem is due to a difference in the behavior of the rl_initialize function between the editline readline emulation and the real GNU libreadline.  Modules/readline.c setup_readline calls several rl functions to create various default bindings, including overriding TAB to "insert" rather than to "trigger completion", then calls rl_initialize allowing the users defaults from .inputrc to override.  It seems the emulated rl_initialize causes all the modified bindings to be discarded, causing TAB to revert to its default "trigger file completion".  The solution in the attached patches is to conditionally call rl_initialize at the beginning of setup_readline, rather than at the end, if the editline emulation is in use.  Patches supplied for py3k and 27 (but not 31 since the feature was never backported there even though it was to 26).  I did not supply any additional tests since I can't think of a straightforward way to simulate the condition in the test framework; suggestions welcome.
msg117288 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-09-24 15:07
Patch looks fine and should IMO be applied
msg117529 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-09-28 15:15
On second thought, the patch isn't quite as harmless as I first thought: the default key-bindings that are created after the call to rl_initialize will replace custom bindings in the users .editrc file.

I've attached a new version of the py3k patch that works around this problem by calling rl_read_init_file(NULL) after setting the default bindings.

This allows me to override the bindings for TAB in ~/.editrc as before.
msg117991 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2010-10-05 06:42
The modified patch looks OK to me and tests OK.  The rl_read_init_file call seems like a reasonable thing for users who are used to using libedit's .editrc.  As a practical matter, though, I think the only thing that would be affected is an .editrc TAB binding.  Some of the initializations done in Modules/readline.c, like rl_bind_key_in_map (for sure) and rl_completer_word_break_characters are silently ignored by the libedit readline-compatibility layer; it does not implement features like the emacs_meta_keymap.
msg124243 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2010-12-17 18:09
I believe this fix should go into 3.2 (and 2.7) as it has been reported by a number of people in various places and the fix risk is low.
msg124272 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-12-18 03:52
Committed to py3k in 87356 and 2.7 in r87358.
Date User Action Args
2010-12-18 03:52:29r.david.murraysetstatus: open -> closed

nosy: + r.david.murray
messages: + msg124272

resolution: fixed
stage: patch review -> resolved
2010-12-17 18:09:11ned.deilysetpriority: normal -> release blocker
nosy: + georg.brandl, benjamin.peterson
messages: + msg124243

2010-11-20 21:37:20ned.deilylinkissue10472 superseder
2010-10-05 06:42:15ned.deilysetmessages: + msg117991
2010-09-28 15:15:14ronaldoussorensetfiles: + issue9907-py3k-ronald.patch

messages: + msg117529
2010-09-24 15:07:04ronaldoussorensetmessages: + msg117288
2010-09-21 09:20:38ned.deilysetfiles: + issue9907-27.patch
2010-09-21 09:19:55ned.deilysetfiles: + issue9907-py3k.patch

versions: - Python 3.1
keywords: + patch
nosy: + zvezdan

messages: + msg117042
stage: patch review
2010-09-20 21:58:44ned.deilysetmessages: + msg116986
2010-09-20 21:36:46ned.deilycreate