classification
Title: The readline module can cause python to segfault
Type: Stage:
Components: Interpreter Core Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: mwh Nosy List: dcoder, mwh
Priority: normal Keywords:

Created on 2005-03-19 21:48 by dcoder, last changed 2005-03-30 11:23 by mwh. This issue is now closed.

Files
File name Uploaded Description Edit
readlineSegfault.py dcoder, 2005-03-19 21:48 Test case
Messages (7)
msg24705 - (view) Author: Yariv Ido (dcoder) Date: 2005-03-19 21:48
When starting an interactive interpreter in another
thread, with readline's completion functionality, there
exists a race condition which causes the Python
interpreter to segfault. There's a small discussion
about this bug at
<http://www.scipy.net/roundup/ipython/issue30>.

Attached is a small proof-of-concept code. Please note
that some people couldn't reproduce this on slower
machines. I've managed to reproduce it on several Linux
systems (Dual Xeon computers), using Python 2.3.4, 2.4
and 2.4.1c2.

Thanks in advance.
msg24706 - (view) Author: Yariv Ido (dcoder) Date: 2005-03-19 23:59
Logged In: YES 
user_id=326689

I may be completely off track here, but shouldn't
on_completion(...) (readline.c) use _PyOS_ReadlineTState
instead of completer_tstate to restore the GIL?

Also, in readline_until_enter_or_signal(...), shouldn't
PyEval_SaveThread()'s return value be saved back to 
_PyOS_ReadlineTState?

It seems that these patches manage to fix the above
segmentation fault...
msg24707 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2005-03-27 20:43
Logged In: YES 
user_id=6656

I'll take a look at this.

Did you submit a bug on the set_startup_hook problem mentioned in the 
ipython tracker?
msg24708 - (view) Author: Yariv Ido (dcoder) Date: 2005-03-28 22:33
Logged In: YES 
user_id=326689

I haven't (Completely forgot about it, after finding the
main one...). Should I post a seperate bug report, or will
this one do?
msg24709 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2005-03-29 12:10
Logged In: YES 
user_id=6656

Unless there's some reason to suspect they share a common
cause (and, I don't see one) a second report is more
appropriate.

Also, if you understand what's going on, you can explain
there :)
msg24710 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2005-03-30 10:10
Logged In: YES 
user_id=6656

While it does seem a bit odd to call the completer function in the thread 
that calls set_completer and not the thread that calls readline(), I don't 
immediately see how this is leading to segfaults.  Do you?

Anyway, I can reproduce the problem, so if your fix fixes it, I'll check it in 
(irrespective of fixing segfaults, it just makes more sense).
msg24711 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2005-03-30 11:23
Logged In: YES 
user_id=6656

Oh, forget all that, I had a leap in understanding about the existing threads-
vs-hooks code in readline.c: it's all insane nonsense.

I used the PyGILState_* API rather than your suggestion, on the very faint 
chance that an embedder might be calling readline() directly and thus have 
not be setting _PyOS_ReadlineTState.

I also fixed the 'checking the return value from a hook function' while I was 
there, so you don't need to submit a new report for that.

Modules/readline.c revision 2.83.

Thanks for the report!
History
Date User Action Args
2005-03-19 21:48:23dcodercreate