classification
Title: PyOS_ReadlineFunctionPointer violates PyMem_Malloc() API: the GIL is not hold
Type: Stage:
Components: Interpreter Core, Library (Lib) Versions: Python 3.3, Python 3.4, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: PyOS_Readline drops GIL and calls PyOS_StdioReadline, which isn't thread safe
View: 16742
Assigned To: Nosy List: vstinner
Priority: normal Keywords: patch

Created on 2013-06-13 21:02 by vstinner, last changed 2013-06-13 22:07 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
readline_gil.patch vstinner, 2013-06-13 21:13 review
Messages (3)
msg191089 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-06-13 21:02
The callback PyOS_ReadlineFunctionPointer (used to read a line from the standard input) must return a buffer allocated by PyMem_Malloc(), but PyOS_Readline() releases the GIL before calling PyOS_ReadlineFunctionPointer.

Simplified extract of PyOS_Readline():

    Py_BEGIN_ALLOW_THREADS
    if (!isatty (fileno (sys_stdin)) || !isatty (fileno (sys_stdout)))
        rv = PyOS_StdioReadline (sys_stdin, sys_stdout, prompt);
    else
        rv = (*PyOS_ReadlineFunctionPointer)(sys_stdin, sys_stdout,
                                             prompt);
    Py_END_ALLOW_THREADS

tok_nextc() calls PyOS_Readline() and calls PyMem_FREE() to release its result.

PyOS_ReadlineFunctionPointer should allocate memory using malloc(), not using PyMem_Malloc(). But PyOS_Readline() should copy the line into a buffer allocated by PyMem_Malloc() to keep backward compatibility.

See also issue #18203 and #3329.
msg191090 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-06-13 21:13
Here is a patch for Python 3.4.
msg191091 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-06-13 21:20
Oh, this is a duplicate of #16742, read also the thread on python-dev:
http://mail.python.org/pipermail/python-dev/2012-December/123225.html
History
Date User Action Args
2013-06-13 22:07:58vstinnersetsuperseder: PyOS_Readline drops GIL and calls PyOS_StdioReadline, which isn't thread safe
2013-06-13 21:20:09vstinnersetstatus: open -> closed
resolution: duplicate
messages: + msg191091
2013-06-13 21:13:55vstinnersetfiles: + readline_gil.patch
keywords: + patch
messages: + msg191090
2013-06-13 21:02:17vstinnercreate