classification
Title: OS X installer variants have confusing readline differences
Type: behavior Stage: resolved
Components: Build, macOS Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: eric.araujo, loewis, ned.deily, ronaldoussoren
Priority: high Keywords:

Created on 2010-12-09 21:44 by ned.deily, last changed 2015-06-06 01:46 by ned.deily. This issue is now closed.

Files
File name Uploaded Description Edit
issue-10666.txt ronaldoussoren, 2013-07-06 09:27 review
Messages (13)
msg123701 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2010-12-09 21:44
32-bit-only OS X installers build and link to a copy of the GNU readline library for use by the readline module in the standard library.  But, the newer 64-bit/32-bit installer variants for 2.7 and 3.2 link to the OS X supplied BSD editline (libedit) library using its GNU readline compatibility interface.  This creates a confusing situation for users: depending on which installer variant is used for 2.7 or 3.2, the commands needed to enable things like tab completion.

Here's a snippet of what I have in my startup file:

    import readline
    import rlcompleter
    if 'libedit' in readline.__doc__:
        readline.parse_and_bind("bind ^I rl_complete")
    else:
        readline.parse_and_bind("tab: complete")

While obviously this can be handled, it seems like an unnecessary burden on users.  I think the primary reason for adding the editline support was to make it simpler for developers building their own Pythons.  That's not a concern for the installer build.  Another concern may have been to avoid shipping a copy of GNU readline which is GPL-licensed.  (The installer currently, to the best of my knowledge, does not document in a README or elsewhere that GNU readline is included.)  That seems a problem.  This disparity also could cause problems elsewhere (see Issue5845).

I see two solutions:

1. (trivial) Change the installer to always build and include GNU readline regardless of SDK (today, editline is used with builds using SDK 10.5 or higher).  

2. (TBD) Build and link with the open-source version of editline for all installers.

In either case, the installer should include license info on included 3rd-party libraries somewhere in the installer README and/or installed files.
msg135467 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-05-07 15:12
FTR, readline.read_init_file() works with libedit too (provided that the config file uses libedit-style format), so you can remove the branch from your code.
msg135500 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-05-07 19:47
́Éric, was your comment in msg135467 intended for another issue?
msg135589 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-05-09 14:28
No.  You wrote this: “Here's a snippet of what I have in my startup file: [code with if 'libedit' in readline.__doc__ complication] While obviously this can be handled, it seems like an unnecessary burden on users.”  My message addresses this: using read_init_file removes the need for this complication (the four lines with the in test).  IOW, I was suggesting that we could add a mention of libedit in the readline docs and advise that people use read_init_file, then close this bug.
msg135614 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-05-09 16:56
Sorry, I don't see how that could help.  The point I was making is that the directives accepted by GNU readline and BSD editline are completely different and one way or another the user is forced to deal with that.  If you used read_init_file, you would still need two different files anyway, no?
msg135616 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-05-09 17:11
Ah, I thought that read_init_file used a different filename with libedit.

Would there be a downside in implementing your suggestion labeled trivial, i.e. always use readline?
msg135618 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-05-09 17:36
Even if used a different file name, you still have to set up two different sets of directives.

The main drawback to the trivial suggestion is that it continues to pull in GNU readline, which is now GPLv3-licensed, into the python.org OS X installers. In general, we try to avoid shipping GPL-licensed software since it can complicate the allowable usages of the installed Python.

(Since I opened this one, I'm reassigning it to myself.)
msg135619 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-05-09 17:43
> Even if used a different file name, you still have to set up two
> different sets of directives.

I was assuming this would not be a problem: the .inputrc (or hypothetical .editrc file) is written once for all applications, it’s not a Python-specific task.  What I didn’t see is that users do not necessarily know about which file is used, so it’s at least a doc problem.

For the perceived problem with licensing, I’m adding Martin to nosy, not because he’s a PSF director but because he’s had to make similar choices for the Windows installers.

Martin, here’s the context: the Mac installer sometimes builds readline with the real readline or libedit.  There’s a doc problem which can be solved either by always using the same lib.  The easiest to implement now would be choosing readline, but Ned says: “The main drawback to the trivial suggestion is that it continues to pull in GNU readline, which is now GPLv3-licensed, into the python.org OS X installers. In general, we try to avoid shipping GPL-licensed software since it can complicate the allowable usages of the installed Python.”  Could you provide advice?
msg135621 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-05-09 17:59
́Éric, Martin was involved in the earlier discussions when the support for editline was originally added to the readline module.  We've been over this ground before.  See Issue6872 and Issue6877.

There are a number of options here.  I plan to investigate the supplying editline with the older installer variant.  That fits in with some other installer improvements that I'm working on.  I would prefer to defer further discussion on this issue until I've had a chance to further pursue that option.
msg135643 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-05-09 20:45
It's true that we should try to avoid distributing GPL'ed code with binary installers. At a minimum, we would have to provide the source of readline for download also, and we would have to warn anybody redistributing our binaries that they must include the source of readline and Python on the distribution media. So it's better to link against libedit, I'd say.
msg135677 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2011-05-10 05:28
FWIW I'm in favor of shipping libedit as well with the 32-bit variant of the installer. The installer for 10.6 can continue to use the system install, that's good enough for to be used.

The download link for the copy in OSX 10.6.7 is <http://opensource.apple.com/tarballs/libedit/libedit-13.tar.gz>, this is BSD licensed code that we can ship in the installer.
msg192392 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2013-07-06 09:27
Download link for the latest version: http://opensource.apple.com/tarballs/libedit/libedit-31.tar.gz

The archive only includes an Xcode based build system, the makefile based system was removed some time ago.

It might be better to use <http://www.thrysoee.dk/editline/libedit-20130611-3.1.tar.gz> instead (which is a distribution of netbsd libedit with a configure script)

I've attached a patch (relative to the tip of the 3.3 branch for now) that teaches build-installer to use a static build of the trysoee distribution of libedit.

The patch is completely untested at the moment, I will be able to build a 10.6/intel installer tonight and testing with the PPC build will have to wait until I get back from EP.

A problem with using libedit unconditionally: we will likely get complaints from users when changing this in a bugfix release, it would be better to only apply this patch to the default branch.
msg244880 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2015-06-06 01:46
As of Python 3.3.0 and 2.7.9, the 32-bit-only OS X installers on python.org now have a minimum deployment target of 10.5, rather than 10.3.  This means that all current OS X installers on python.org are now dynamically linked with the Apple-provided system editline (libedit) libraries.  Users who prefer or need to use the GPL-licensed GNU readline libraries can pip install the third-party gnureadline from PyPI (https://pypi.python.org/pypi/gnureadline).
History
Date User Action Args
2015-06-06 01:46:49ned.deilysetstatus: open -> closed
resolution: out of date
messages: + msg244880

stage: needs patch -> resolved
2013-07-06 09:27:41ronaldoussorensetfiles: + issue-10666.txt

messages: + msg192392
2011-05-10 05:28:38ronaldoussorensetmessages: + msg135677
2011-05-09 20:45:54loewissetmessages: + msg135643
2011-05-09 17:59:42ned.deilysetmessages: + msg135621
2011-05-09 17:43:48eric.araujosetnosy: + loewis
messages: + msg135619
2011-05-09 17:36:36ned.deilysetassignee: ronaldoussoren -> ned.deily
messages: + msg135618
2011-05-09 17:11:11eric.araujosetmessages: + msg135616
2011-05-09 16:56:27ned.deilysetmessages: + msg135614
2011-05-09 14:28:59eric.araujosetmessages: + msg135589
2011-05-07 19:47:39ned.deilysetmessages: + msg135500
2011-05-07 15:12:44eric.araujosetnosy: + eric.araujo
messages: + msg135467
2010-12-09 21:44:22ned.deilycreate