New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
enable compilation of readline module on Mac OS X 10.5 and 10.6 #51126
Comments
The attached patch enables compilation and use of the readline module on I used the patch for almost two years already with Python 2.4 and since The patch is written in such a way that it does *not* affect the With the readline module compiled in, it is enough to put in .editrc
and one gets a vi emulation in the python interactive interpreter. You can also try it directly from the shell:
It would be nice if we could get this included into Python-2.6.3 |
Changed type to "crash" because compilation of readline module without |
When testing the patch make sure that your readline module has been Assuming --prefix=${HOME}/opt, the output of
should contain /usr/lib/libedit.2.dylib. |
I'm +1 on merging this functionality. See also: bpo-6872 As I mentioned there we should ensure that readline linked to libedit It would also be nice if one could programmaticly detect if readline is BTW. I'm pretty sure that the readline emultation on Leopard was pretty |
There's this rl_library_version constant defined in editline/readline C Regarding your comment about editline being broken on Leopard: The One person used it with Python 2.4 and ipython. He did not have a Perhaps you meant it was broken on Tiger (10.4; Darwin 8). |
The attached patch is a slightly cleaner version of your patch. What I My changes w.r.t. your patch:
The patch seems to work on fine, including browsing in ipython's history PS. I mentioned ipython because I know there were issues with ipython Marking this issue as 'needs review' for two reasons:
|
The patch
It fails to compile
function) The fixed patch is attached as There are also a few minor stylistic nits I took a liberty to fix. In readline.c Line 496 Line 505
Lines 1085/86 Line 1070 In setup.py Line 561
Just out of curiosity: does anyone know why the first if statement that |
Do the remove_history_item and replace_history_item functions also need |
Mark: yes those functions need to be changed as well. |
Mark: it turns out that GNU readline has a rather odd interface, only I've attached a very minimal testcase for the history manipulation The tests now also pass using libedit emulation (after fixing the I'll file a new issue about the readline documentation, that's unrelated |
Committed my latest version of the patch as r74970 (trunk) and r74971 Barry: what's your opinion on a backport of this to 2.6? |
Is this open or closed? Wondering as I just updated my checkout and I am
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000
0x00000001003effa6 in call_readline (sys_stdin=0x7fff70a230c0,
sys_stdout=0x7fff70a23158, prompt=0x1005da194 ">>> ") at readline.c:1029
1029 line = history_get(state->length
- 1)->line;
(gdb) bt
#0 0x00000001003effa6 in call_readline (sys_stdin=0x7fff70a230c0,
sys_stdout=0x7fff70a23158, prompt=0x1005da194 ">>> ") at readline.c:1029
#1 0x00000001000044d8 in PyOS_Readline (sys_stdin=0x7fff70a230c0,
sys_stdout=0x7fff70a23158, prompt=0x1005da194 ">>> ") at
myreadline.c:208
#2 0x0000000100006ce0 in tok_nextc (tok=0x100890c10) at tokenizer.c:781
#3 0x0000000100005464 in tok_get (tok=0x100890c10,
p_start=0x7fff5fbfe9e8, p_end=0x7fff5fbfe9e0) at tokenizer.c:1128
#4 0x00000001000053a4 in PyTokenizer_Get (tok=0x100890c10,
p_start=0x7fff5fbfe9e8, p_end=0x7fff5fbfe9e0) at tokenizer.c:1568
#5 0x000000010000388b in parsetok (tok=0x100890c10, g=0x100251f68,
start=256, err_ret=0x7fff5fbfeb10, flags=0x7fff5fbfeb0c) at
parsetok.c:159
#6 0x0000000100003fe8 in PyParser_ParseFileFlagsEx (fp=0x7fff70a230c0,
filename=0x1001fba28 "<stdin>", g=0x100251f68, start=256,
ps1=0x1005da194 ">>> ", ps2=0x1005da2b4 "... ", err_ret=0x7fff5fbfeb10,
flags=0x7fff5fbfeb0c) at parsetok.c:106
#7 0x0000000100193b76 in PyParser_ASTFromFile (fp=0x7fff70a230c0,
filename=0x1001fba28 "<stdin>", start=256, ps1=0x1005da194 ">>> ",
ps2=0x1005da2b4 "... ", flags=0x7fff5fbfeea0, errcode=0x7fff5fbfebec,
arena=0x1004427d0) at pythonrun.c:1461
#8 0x00000001001937c9 in PyRun_InteractiveOneFlags (fp=0x7fff70a230c0,
filename=0x1001fba28 "<stdin>", flags=0x7fff5fbfeea0) at pythonrun.c:823
#9 0x0000000100193091 in PyRun_InteractiveLoopFlags (fp=0x7fff70a230c0,
filename=0x1001fba28 "<stdin>", flags=0x7fff5fbfeea0) at pythonrun.c:763
#10 0x0000000100192db8 in PyRun_AnyFileExFlags (fp=0x7fff70a230c0,
filename=0x1001fba28 "<stdin>", closeit=0, flags=0x7fff5fbfeea0) at
pythonrun.c:732
#11 0x00000001001af808 in Py_Main (argc=1, argv=0x7fff5fbfef40) at
main.c:603
#12 0x0000000100001325 in main (argc=1, argv=0x7fff5fbfef40) at
python.c:23 |
This should have been closed, although readline shouldn't crash either. Brett: What version of OSX do you use? Readline works fine for me on OSX BTW. The crashlog indicates you are no longer using GNU readline, but This is one thing we completely failed to look at: how to determine It may be better to either add a configure flag to explicitly select |
Brett, what does this command return for you? otool -L /path/to/lib/python2.4/lib-dynload/readline.so |
Make that python2.6 in the command above. :-) |
Better yet: otool -vL $(python -c 'import readline; print readline.__file__') (Replace "python" by the interpreter that your actually using). I'm still interested to know the OS release as well. |
I've tested with readline 6 on OSX 10.6 as well. Both that and the system The current behaviour for me:
|
I assume you had to pass extra -I and -L flags when compiling with GNU |
Zvezdan: I did not have to pass additional arguments to get readline, the I'd love to have a way to restrict the default compiler and linker search |
I'm on OS X 10.6 and I *thought* I was using GNU Readline 6. But if my I'm going to go ahead and close this as you got it build and working for |
Brett, IMO, your backtrace only implies that readline module was built However, the following scenario is possible. Some packaging tools Because of the way setup.py stashes /usr/local/lib first in the path, This is just a wild guess of course. That's why I was interested in the output of otool command on your build |
The readline-libedit-2.patch no longer applies against the 2.6 branch because of changes in setup.py since then. If this is fixed and the subsequent patch is reviewed and approved, then this can be landed for Python 2.6.5. |
release blocker for 2.6.5 |
The readline-libedit-2.6.5.patch is attached. The patch was applied and python built in several configurations on Mac Can somebody else test on Mac OS X 10.5 (Leopard)? 32-bit Configuration with system libedit::
Configuration with GNU readline 6.1::
Failed tests for both builds:
64-bit Configuration with system libedit::
Configuration with GNU readline 6.1::
Failed tests for both builds:
Unexpected skips:
Universal Configuration with system libedit::
Configuration with GNU readline 6.1::
Failed tests for both builds:
Unexpected skips:
when run as 64-bit or 32-bit executable. Errors in both asynchat and smtplib are caused by the same issue in
but this is not caused by readline patch. |
I forgot to add that the patch for 2.6.5 is based on: |
This worked fine on OS X 10.6, but failed on OS X 10.5: gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -I/Users/barry/projects/python/python26/./Include -I/Users/barry/projects/python/python26/./Mac/Include -I. -IInclude -I./Include -I/usr/local/include -I/Users/barry/projects/python/python26/Include -I/Users/barry/projects/python/python26 -c /Users/barry/projects/python/python26/Modules/readline.c -o build/temp.macosx-10.3-ppc-2.6/Users/barry/projects/python/python26/Modules/readline.o Failed to find the necessary bits to build these modules: Failed to build these modules: running build_scripts |
This is a configuration bug: /Users/barry/projects/python/python26/Modules/readline.c -o build/temp.macosx-10.3-ppc-2.6 ... Why is it trying to build using macosx-10.3 target? This is a bug in configure.in |
It's not a configuration bug. All current python.org python installers are built with MACOSX_DEPLOYMENT_TARGET=10.3 and with the 10.4u SDK to allow one installer to work on systems from 10.3.9 through 10.6. |
Barry, I'm sorry, I only now noticed this line in your compilation on 10.5: /Users/barry/projects/python/python26/Modules/readline.c:41: error: conflicting types for 'completion_matches' This problem was introduced by a patch in bpo-4204. Please notice that my original patch attached here (readline-trunk.patch) has this part in it, which is very important for 10.5: @@ -6,6 +6,7 @@
/* Standard definitions */
#include "Python.h"
+#include <stdlib.h>
#include <setjmp.h>
#include <signal.h>
#include <errno.h> Ronald has omitted that part when he was changing the patch and I completely forgot about its importance since I'm running 10.6 and did not have problems. Initially my first version of the patch was checking for __APPLE__ and avoided to re-define completion_matches on lines 35-36. I used that patch when 2.6.1 came out in my personal builds. Then I found out that including stdlib early avoided the problem. I really do not have a 10.5 machine to test on now. If yes, I'd be glad to make a new patch against 2.6 branch and trunk/3.x branch (probably need the same include line). If not, then we need to make a choice of #ifdef __FreeBSD__ as I suggested in bpo-4204, or #ifndef __APPLE__ as I used in my first personal version of the patch. The problem with re-definition of completion_matches did not exist in 2.4 and 2.5 and is definitely introduced by a patch in bpo-4204, which annoyed me because it broke a modern OS to support a very old version of FreeBSD (4.x). <aside note> We had a discussion on pythonmac-sig about it and people mostly agreed but I do not remember if any issue was opened in the tracker to act on it or not. |
This does not work out of trunk for me: euclid:trunk minge$ sw_vers --- Modules/readline.c (revision 78019)
+++ Modules/readline.c (working copy)
@@ -6,6 +6,7 @@
/* Standard definitions */
#include "Python.h"
+#include <stdlib.h>
#include <setjmp.h>
#include <signal.h>
#include <errno.h>
euclid:trunk minge$ make Modules/Setup.dist is newer than Modules/Setup; Python build finished, but the necessary bits to build these modules were not found: Failed to build these modules: running build_scripts I am more than happy to run more tests with respect to this issue. I am tired of seeing this build break everyday :) |
@meador: confirmed. This does not fix the 10.5 build. |
In the off chance that this same problem were to occur on another platform, wouldn't be better to test for 'completion_matches' in configure? Perhaps something like the attached patch? (fixes the issue for me on OS X 10.5 in the Python trunk) |
OK, so bpo-4204 patch is causing problems. I'm attaching a new patch for 2.6.5 (2.6 branch) that uses the __APPLE__ check. |
The patch with the same correction against trunk is attached so that Meador can test it. |
Meador, I just looked at your patch (it seems we were posting patches at about the same time). That's a good fix too. Barry it's your call. |
re: msg98913 The addition of an #include of stdlib.h is unnecessary because Python.h already does that. re: msg98908 Building against with a 10.3 deployment target should be harmless. A universal build targets 10.3 on purpose because that results in binaries that work on all supported OSX platforms. BTW. I've pruned the list of python versions, this patch will never be applied to 2.4. BTW2. I'll do the backport later today. |
Zvezdan, your "fix1" patch works perfectly for me in Python 2.6 on both OS X 10.5 and 10.6. Great work! I approve applying this patch for Python 2.6.5. Ronald, would you like to do the honors? |
I have a question about this bit of the patch: @@ -38,10 +38,31 @@
#if defined(_RL_FUNCTION_TYPEDEF)
extern char **completion_matches(char *, rl_compentry_func_t *);
#else
+#if !defined(__APPLE__)
extern char **completion_matches(char *, CPFunction *);
#endif
#endif
+#endif That bit is not in the trunk, should it be forward ported to the trunk? The actual prototype on 10.5 and 10.6 is: char **completion_matches(const char *, CPFunction *); Wouldn't it be better to change the prototype in readline.c to match? As that's not a critical change I've committed the fix1 patch as is in r78096. |
FWIW, I would really like to have it.
I may have missed something, but the patch is actually *excluding* the On the other hand, if you are proposing to make them 'match' just to avoid Regards, -- Meador |
Ronald,
Yes, that should be applied to trunk and 3.x to make it work on Mac OS X 10.5 (Leopard). I indicated that in msg98979. The explanation why that part is needed is given in msg98978 and msg98913. We must *avoid* the duplicate declaration on Mac OS X, because the same declaration is already in /usr/include/readline/readline.h. So, the readline-libedit-trunk.patch I attached yesterday should be applied to trunk (2.7) and 3.x.
It seems that you forgot to
before committing. P.S. |
I've added the tests to the 2.6 branch and have ported the #ifdef guard around the prototype for completion_matches to the trunk and 3.2. I'm therefore closing the issue. |
On Thu, Feb 11, 2010 at 7:16 AM, Ronald Oussoren <report@bugs.python.org>wrote:
Verified in trunk. Thanks Ronald! |
See bpo-8066 for additional patches to correct problems when building on 10.5 or 10.6 for deployment targets of 10.4 or earlier. |
Also note that this feature has not been backported to 3.1 which means it will not be in the upcoming 3.1.2 release. (It will not be an issue for the python.org 3.1.2 installer which will be built with GNU readline as usual to support older systems.) |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: