classification
Title: Failure to compile with readline-6.3-rc1
Type: compile error Stage: resolved
Components: Extension Modules Versions: Python 3.4, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: Arfrever, benjamin.peterson, georg.brandl, jhl, koobs, larry, ned.deily, python-dev, serhiy.storchaka
Priority: release blocker Keywords: patch

Created on 2014-01-24 04:26 by jhl, last changed 2014-09-30 17:37 by georg.brandl. This issue is now closed.

Files
File name Uploaded Description Edit
readline_func_cast.patch serhiy.storchaka, 2014-01-24 11:30 review
issue20374_no_cast_33.patch ned.deily, 2014-02-06 00:42 2.7 and 3.3
issue20374_no_cast_3x.patch ned.deily, 2014-02-06 00:42 default review
Messages (20)
msg209035 - (view) Author: jhl (jhl) Date: 2014-01-24 04:26
Python 2.7.6 does not compile with readline 6.3 (rc1).
It looks like the RL_FUNCTION_TYPEDEF support in Modules/readline.c is not quite complete.

The following patch works for me, both for 2.7.6 and 3.3.3.

--- a/Modules/readline.c	2013-11-10 18:36:41.000000000 +1100
+++ b/Modules/readline.c	2014-01-24 15:11:04.182417214 +1100
@@ -911,12 +911,27 @@
     rl_bind_key_in_map ('\t', rl_complete, emacs_meta_keymap);
     rl_bind_key_in_map ('\033', rl_complete, emacs_meta_keymap);
     /* Set our hook functions */
-    rl_startup_hook = (Function *)on_startup_hook;
+    rl_startup_hook =
+#if defined(_RL_FUNCTION_TYPEDEF)
+        (rl_hook_func_t *)on_startup_hook;
+#else
+        (Function *)on_startup_hook;
+#endif
 #ifdef HAVE_RL_PRE_INPUT_HOOK
-    rl_pre_input_hook = (Function *)on_pre_input_hook;
+    rl_pre_input_hook = 
+#if defined(_RL_FUNCTION_TYPEDEF)
+        (rl_hook_func_t *)on_pre_input_hook;
+#else
+        (Function *)on_pre_input_hook;
+#endif
 #endif
     /* Set our completion function */
-    rl_attempted_completion_function = (CPPFunction *)flex_complete;
+    rl_attempted_completion_function =
+#if defined(_RL_FUNCTION_TYPEDEF)
+        (rl_completion_func_t *)flex_complete;
+#else
+        (CPPFunction *)flex_complete;
+#endif
     /* Set Python word break characters */
     completer_word_break_characters =
         rl_completer_word_break_characters =
msg209038 - (view) Author: Roundup Robot (python-dev) Date: 2014-01-24 05:33
New changeset 79b82ebc4fd1 by Benjamin Peterson in branch '2.7':
use new readline function types (closes #20374)
http://hg.python.org/cpython/rev/79b82ebc4fd1

New changeset fb2259d9f6b4 by Benjamin Peterson in branch '3.3':
use new readline function types (closes #20374)
http://hg.python.org/cpython/rev/fb2259d9f6b4

New changeset eb251e3624df by Benjamin Peterson in branch 'default':
merge 3.3 (#20374)
http://hg.python.org/cpython/rev/eb251e3624df
msg209055 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-01-24 09:40
The checked-in fixes break builds on OS X that use the Apple-supplied libedit readline compatibility layer.  See, for example:  http://buildbot.python.org/all/builders/AMD64%20Snow%20Leop%203.x/builds/1077/steps/compile/logs/stdio

gcc -fno-strict-aliasing -Werror=declaration-after-statement -g -O0 -Wall -Wstrict-prototypes -I./Include -I. -IInclude -I/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Include -I/Users/buildbot/buildarea/3.x.murray-snowleopard/build -c /Users/buildbot/buildarea/3.x.murray-snowleopard/build/Modules/readline.c -o build/temp.macosx-10.6-x86_64-3.4-pydebug/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Modules/readline.o
/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Modules/readline.c: In function ‘setup_readline’:
/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Modules/readline.c:1001: error: ‘rl_hook_func_t’ undeclared (first use in this function)
/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Modules/readline.c:1001: error: (Each undeclared identifier is reported only once
/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Modules/readline.c:1001: error: for each function it appears in.)
/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Modules/readline.c:1001: error: expected expression before ‘)’ token
/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Modules/readline.c:1003: error: expected expression before ‘)’ token
/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Modules/readline.c:1006: error: ‘rl_completion_func_t’ undeclared (first use in this function)
/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Modules/readline.c:1006: error: expected expression before ‘)’ token
msg209067 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-24 11:30
What if remove casts at all? Ned, is it compiled with libedit?
msg209095 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-01-24 16:20
Serhiy, I'm not sure I understand your question.  libedit includes a GNU readline compatibility layer and that is what _readline.so builds and links with on OS X systems (with an ABI of 10.5 and later) since OS X does not ship with GNU readline.  This is also the case on some BSD systems.  With your patch, _readline.so again builds correctly.
msg209096 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-24 16:41
> With your patch, _readline.so again builds correctly.

Yes, this is what I wanted to know. Sorry for the awkward formulation.

Type casting is dangerous because it may hide the signature mismatch, which 
can cause problems later in run time. It is better to rely on static type 
checking if possible.
msg209097 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2014-01-24 16:41
Okay then. Off we go.
msg209098 - (view) Author: Roundup Robot (python-dev) Date: 2014-01-24 16:44
New changeset 5e42e5764ac6 by Benjamin Peterson in branch '2.7':
new plan: just remove typecasts (closes #20374)
http://hg.python.org/cpython/rev/5e42e5764ac6

New changeset fc62fcd8e990 by Benjamin Peterson in branch '3.3':
new plan: just remove typecasts (closes #20374)
http://hg.python.org/cpython/rev/fc62fcd8e990

New changeset 3c3624fec6c8 by Benjamin Peterson in branch 'default':
merge 3.3 (#20374)
http://hg.python.org/cpython/rev/3c3624fec6c8
msg209225 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-01-25 18:05
FYI, removing the cast causes the following new warnings when compiling 3.3 with gcc-4.2 on OS X 10.5 and 10.6 (haven't checked elsewhere):

Modules/readline.c: In function 'setup_readline':
Modules/readline.c:939: warning: assignment from incompatible pointer type
Modules/readline.c:941: warning: assignment from incompatible pointer type
msg209226 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2014-01-25 18:17
It doesn't complain on Linux. I suppose if we don't want any warnings, we'd have to do something like the originally proposed patch.
msg209227 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-25 18:29
I'm surprised that warnings are emitted at lines 939 and 941, but not 944.

I think that instead type casting, the more robust way is to change hook functions signatures for on_startup_hook and on_pre_input_hook.

static int
#ifdef _RL_FUNCTION_TYPEDEF /* or may be test libedit macro? */
on_startup_hook(void)
#else
on_startup_hook()
#endif
msg210355 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-02-06 00:42
Using Serhiy's suggestion, here are patches for 2.7/3.3 and default that condition the hook function signatures.  With these patches, all three branches compile without warnings and pass test_readline using OS X libedit, readline 6.3-rc2, and an older readline.
msg210356 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2014-02-06 00:44
That is sure ugly, but okay. :)
msg210357 - (view) Author: Roundup Robot (python-dev) Date: 2014-02-06 00:55
New changeset 0b5b0bfcc7b1 by Ned Deily in branch '2.7':
Issue #20374: Avoid compiler warnings when compiling readline with libedit.
http://hg.python.org/cpython/rev/0b5b0bfcc7b1

New changeset 9131a9edcac4 by Ned Deily in branch '3.3':
Issue #20374: Avoid compiler warnings when compiling readline with libedit.
http://hg.python.org/cpython/rev/9131a9edcac4

New changeset 0abf103f5559 by Ned Deily in branch 'default':
Issue #20374: merge
http://hg.python.org/cpython/rev/0abf103f5559
msg210358 - (view) Author: Roundup Robot (python-dev) Date: 2014-02-06 01:04
New changeset 74c7a6fd8b45 by Ned Deily in branch '2.7':
Issue #20374: delete spurious empty line
http://hg.python.org/cpython/rev/74c7a6fd8b45

New changeset 6616c94d6149 by Ned Deily in branch '3.3':
Issue #20374: delete spurious empty line
http://hg.python.org/cpython/rev/6616c94d6149

New changeset 0b91e764b889 by Ned Deily in branch 'default':
Issue #20374: merge
http://hg.python.org/cpython/rev/0b91e764b889
msg210868 - (view) Author: Roundup Robot (python-dev) Date: 2014-02-10 21:04
New changeset a7e048674fef by Ned Deily in branch '3.3':
Issue #20374: Avoid compiler warnings when compiling readline with libedit.
http://hg.python.org/cpython/rev/a7e048674fef

New changeset de02d414590d by Ned Deily in branch '3.3':
Issue #20374: delete spurious empty line
http://hg.python.org/cpython/rev/de02d414590d
msg227954 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-09-30 17:17
The readline module no longer compiled in 3.2. May be we should apply these patches to 3.2.
msg227955 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-09-30 17:29
This isn't a security issue and in general we do not make changes to branches in security-fix mode to support new releases of external components or operating system releases.  There are plenty of other fixes that should be ported to 3.2 if we go down that path.  So I'm closing this again unless Georg disagrees.
msg227962 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2014-09-30 17:35
+1 for backporting build fixes for major platforms (e.g. fix for this issue).
msg227964 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-09-30 17:37
I agree with Ned. 3.2 is not maintained anymore, we only provide security fixes.

(Exceptions prove the rule.  One exception I committed today was to stop test_urllibnet from failing, so that I could better see real failures on the buildbots.)
History
Date User Action Args
2014-09-30 17:37:56georg.brandlsetmessages: + msg227964
2014-09-30 17:35:09Arfreversetmessages: + msg227962
2014-09-30 17:29:30ned.deilysetstatus: open -> closed

messages: + msg227955
2014-09-30 17:17:44serhiy.storchakasetstatus: closed -> open

messages: + msg227954
2014-03-01 13:13:40koobssetnosy: + koobs
2014-02-10 21:04:40python-devsetmessages: + msg210868
2014-02-06 01:05:52ned.deilysetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2014-02-06 01:04:28python-devsetmessages: + msg210358
2014-02-06 00:55:51python-devsetmessages: + msg210357
2014-02-06 00:44:31benjamin.petersonsetassignee: benjamin.peterson -> ned.deily
messages: + msg210356
2014-02-06 00:42:26ned.deilysetfiles: + issue20374_no_cast_3x.patch
2014-02-06 00:42:09ned.deilysetstatus: closed -> open
files: + issue20374_no_cast_33.patch
messages: + msg210355

resolution: fixed -> (no value)
stage: resolved -> patch review
2014-01-25 18:29:38serhiy.storchakasetmessages: + msg209227
2014-01-25 18:17:14benjamin.petersonsetmessages: + msg209226
2014-01-25 18:05:56ned.deilysetmessages: + msg209225
2014-01-24 16:44:49python-devsetstatus: open -> closed
resolution: fixed
messages: + msg209098

stage: needs patch -> resolved
2014-01-24 16:41:17benjamin.petersonsetmessages: + msg209097
2014-01-24 16:41:14serhiy.storchakasetmessages: + msg209096
2014-01-24 16:20:30ned.deilysetmessages: + msg209095
2014-01-24 11:30:27serhiy.storchakasetfiles: + readline_func_cast.patch

nosy: + serhiy.storchaka
messages: + msg209067

keywords: + patch
2014-01-24 09:40:59ned.deilysetstatus: closed -> open

assignee: benjamin.peterson

nosy: + ned.deily
messages: + msg209055
resolution: fixed -> (no value)
stage: resolved -> needs patch
2014-01-24 05:33:39python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg209038

resolution: fixed
stage: resolved
2014-01-24 04:35:07Arfreversetpriority: normal -> release blocker
nosy: + georg.brandl, Arfrever, larry, benjamin.peterson

versions: + Python 3.4
2014-01-24 04:27:22jhlsetversions: + Python 3.3
2014-01-24 04:27:04jhlsettype: compile error
2014-01-24 04:26:50jhlcreate