When debugging our project I find something interesting. In PyOS_Readline it releases the GIL and delegate its job to PyOS_ReadlineFunctionPointer, which could be call_readline or PyOS_StdioReadline(I don't find where vms__StdioReadline is defined). But in the two functions, they use some C APIs like PyMem_Malloc/MALLOC, PyErr_SetString which need guarded by GIL. I don't understand why not doing find-grained lock control in call_readline or PyOS_StdioReadline since the code is ancient. :-(

I find this because our project makes test_cmd_line fail with `echo "Timer\n" | python -i -m timeit -n 1`.

AFAIK PyMem_Malloc/MALLOC can be used with released GIL.

In pymem.h, it's stated that "The GIL must be held when using these APIs". I think the reason is when PYMALLOC_DEBUG is defined, they finally delegate to PyObject_Malloc.

Victor must be familiar with them since once he proposed to remote the restriction: https://mail.python.org/pipermail/python-dev/2013-June/126822.html.

You must hold the GIL to call PyMem_Malloc(). A debug assert should be
raised if you don't hold the GIL since Python 3.6 with PYTHONMALLOC=debug.

Call PyMem_RawMalloc().

I fixed Python 3, no?

For Python 2, in practice you can call PyMem_Malloc() without holding the
GIL, it's just malloc() which is thread safe.

>>> I fixed Python 3, no?

Yes. In Python3 they are replaced by PyMem_RawMalloc. But it's not only PyMem_Malloc, there are also PyErr_SetString, PyErr_NoMemory, even in Python3.

BTW, even in Python3, when memory allocators are in debug mode, it finally calls bumpserialno, which IIUC, is not thread safe. But of course it's another issue.

>>> For Python 2, in practice you can call PyMem_Malloc() without holding the GIL, it's just malloc() which is thread safe.

Hmm, I know it. But it's not stated in the doc they are thread safe, I am not sure assuming this is suitable. An example is https://github.com/psycopg/psycopg2/issues/110.

I fixed this issue in bpo-40826, especially with this change:

commit c353764fd564e401cf47a5d9efab18c72c60014e
Author: Victor Stinner <vstinner@python.org>
Date:   Mon Jun 1 20:59:35 2020 +0200

    bpo-40826: Fix GIL usage in PyOS_Readline() (GH-20579)
    
    Fix GIL usage in PyOS_Readline(): lock the GIL to set an exception.
    
    Pass tstate to my_fgets() and _PyOS_WindowsConsoleReadline(). Cleanup
    these functions.