classification
Title: Malformed PyImport_Inittab after re-initialization
Type: Stage: resolved
Components: C API, Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: kryheb, vstinner
Priority: normal Keywords: patch

Created on 2021-06-17 10:14 by kryheb, last changed 2021-06-24 08:48 by kryheb. This issue is now closed.

Files
File name Uploaded Description Edit
inittab-bug.c kryheb, 2021-06-17 10:14 Source code to reproduce an issue
inittab-bug_no-threads.c kryheb, 2021-06-23 08:27 Source code to reproduce an issue
Pull Requests
URL Status Linked Edit
PR 26767 closed kryheb, 2021-06-17 10:21
PR 26874 merged vstinner, 2021-06-23 10:26
PR 26877 merged vstinner, 2021-06-23 12:16
PR 26878 merged vstinner, 2021-06-23 12:21
Messages (10)
msg395985 - (view) Author: kryheb (kryheb) * Date: 2021-06-17 10:14
Hi all,

I observed misbehavior trying to embed the Python interpreter into a C app.
It seems that after re-initialization, PyImport_Inittab is malformed and points to the memory freed _PyImport_Fini2.


Steps to reproduce:

1. Append embedded module
2. Initialize Python from config with run_filename
3. Run main with an infinite loop
4. Interrupt script execution with async call
5. Finalize Python
6. Repeat all above

Observed behavior:
The script is executed at first iteration, but re-initialization fails with an error:
"
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1187, in _install_external_importers
ModuleNotFoundError: No module named 'posix'

Error: external importer setup failed
"

Head of modules list at fist run:
------ Modules: ------
#0 'posix'
#1 'errno'
#2 'pwd'
----------------------
and after re-initialization:
------ Modules: ------
#0 'P ''
#1 'errno'
#2 'pwd'
----------------------

An issue discovered on:
Fedora 33
gcc (GCC) 10.3.1 20210422 (Red Hat 10.3.1-1)
python3-devel.x86_64 3.9.5-2.fc33

Issue still exists on the latest main and on the rc 3.10
Source code to reproduce an issue in attachment.

Best regards,
Krystian Heberlein
msg396300 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-21 23:26
inittab-bug.c uses the Python C API in 3 threads:

* Thread A (init_proc) calls Py_InitializeFromConfig()
* Thread B (run_proc) calls Py_RunMain()
* The main thread (main) calls Py_FinalizeEx()

Problem: the thread B (run_proc) doesn't hold the GIL and so I get a fatal error with Python built in debug mode:
------------
$ gcc inittab-bug.c -ggdb -pthread -lpython3.11d -L. -I. -I Include/
$ PYTHONPATH=$PWD/Lib LD_LIBRARY_PATH=$PWD ./a.out
------ Modules: ------
#0 'posix'
#1 'errno'
#2 'pwd'
----------------------
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Run Python Script
Fatal Python error: _PyMem_DebugMalloc: Python memory allocator called without holding the GIL
Python runtime state: initialized

Thread 0x00007f61ce5c1640 (most recent call first):
<no Python frame>
Abandon (core dumped)
------------

If I remove the thread B (comment the comment), I get a similar error in the main thread which calls Py_FinalizeEx().

Please fix your usage of the GIL. For example, you can try to use:

* https://docs.python.org/dev/c-api/init.html#c.PyGILState_Ensure
* https://docs.python.org/dev/c-api/init.html#c.PyGILState_Release

Usually, Python initialization and Python finalization is done in the same thread, but another thread can use the Python C API if it acquires the GIL using PyGILState_Ensure().
msg396385 - (view) Author: kryheb (kryheb) * Date: 2021-06-23 08:27
Hi vstinner,

I apologize for the inaccurate code sample.
Please find a simplified example without threads `inittab-bug_no-threads.c` in attachments. Please notice that __main__.py has changed, there is no infinite loop anymore.

Steps to reproduce:

1. Append embedded module
2. Initialize Python from config with run_filename
3. Run main
4. Finalize
5. Repeat all above

Observed behavior:
The script is executed at the first iteration, but re-initialization fails with a segmentation fault:

gdb) r
Starting program: /home/kheb/proj/tmp/pyc/a.out 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Run Python Script
>>> This is the python script
Python Script completed (0)

Program received signal SIGSEGV, Segmentation fault.
__strlen_avx2 () at ../sysdeps/x86_64/multiarch/strlen-avx2.S:65
65		VPCMPEQ (%rdi), %ymm0, %ymm1
Missing separate debuginfos, use: dnf debuginfo-install libxcrypt-4.4.20-2.fc33.x86_64
(gdb) bt
#0  __strlen_avx2 () at ../sysdeps/x86_64/multiarch/strlen-avx2.S:65
#1  0x00007ffff7d039f0 in PyUnicode_FromString (
    u=0xdddddddddddddddd <error: Cannot access memory at address 0xdddddddddddddddd>)
    at Objects/unicodeobject.c:2309
#2  0x00007ffff7dbf049 in list_builtin_module_names () at ./Python/sysmodule.c:2056
#3  0x00007ffff7dc1777 in _PySys_InitCore (tstate=tstate@entry=0x422ac0, sysdict=sysdict@entry=0x7fffea221070)
    at ./Python/sysmodule.c:2813
#4  0x00007ffff7dc36ef in _PySys_Create (tstate=tstate@entry=0x422ac0, sysmod_p=sysmod_p@entry=0x7fffffffd9d8)
    at ./Python/sysmodule.c:3087
#5  0x00007ffff7da9b32 in pycore_interp_init (tstate=0x422ac0) at Python/pylifecycle.c:824
#6  0x00007ffff7da9c8e in pyinit_config (runtime=runtime@entry=0x7ffff7fbe820 <_PyRuntime>, 
    tstate_p=tstate_p@entry=0x7fffffffdca8, config=config@entry=0x7fffffffdac0) at Python/pylifecycle.c:866
#7  0x00007ffff7daba3e in pyinit_core (runtime=runtime@entry=0x7ffff7fbe820 <_PyRuntime>, 
    src_config=src_config@entry=0x7fffffffdd40, tstate_p=tstate_p@entry=0x7fffffffdca8) at Python/pylifecycle.c:1029
#8  0x00007ffff7dabb18 in Py_InitializeFromConfig (config=0x7fffffffdd40) at Python/pylifecycle.c:1214
#9  0x00000000004012d3 in main (argc=1, argv=0x7fffffffdfc8) at inittab-bug_no-threads.c:45

I hope this example helps.

Best regards,
Krystian
msg396404 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-23 10:27
Thanks for inittab-bug_no-threads.c reproducer. I managed to reproduce the issue and I wrote PR 26767 to fix with a regression test.
msg396405 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-23 10:27
Oh, I didn't notice your PR 26767 fix! Thanks. I wrote PR 26874 which includes the same fix, but adds also a regression test.
msg396414 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-23 12:13
New changeset 489699ca05bed5cfd10e847d8580840812b476cd by Victor Stinner in branch 'main':
bpo-44441: _PyImport_Fini2() resets PyImport_Inittab (GH-26874)
https://github.com/python/cpython/commit/489699ca05bed5cfd10e847d8580840812b476cd
msg396427 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-23 15:47
New changeset 5ed7827b1620f5b3729bc9767158d24a33863fa9 by Victor Stinner in branch '3.9':
bpo-44441: _PyImport_Fini2() resets PyImport_Inittab (GH-26874) (GH-26878)
https://github.com/python/cpython/commit/5ed7827b1620f5b3729bc9767158d24a33863fa9
msg396428 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-23 15:47
New changeset ece3841d3ddf38875b997eb919488cbb911a66e2 by Victor Stinner in branch '3.10':
bpo-44441: _PyImport_Fini2() resets PyImport_Inittab (GH-26874) (GH-26877)
https://github.com/python/cpython/commit/ece3841d3ddf38875b997eb919488cbb911a66e2
msg396429 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-06-23 15:48
Thanks kryheb for your bug report, reproducer and your fix!

The bug should now be fixed in 3.9, 3.10 and main branches.
msg396462 - (view) Author: kryheb (kryheb) * Date: 2021-06-24 08:48
Thanks vstinner for your help and for taking care of this case!
History
Date User Action Args
2021-06-24 08:48:55kryhebsetmessages: + msg396462
2021-06-23 15:48:43vstinnersetstatus: open -> closed
versions: + Python 3.11
messages: + msg396429

resolution: fixed
stage: patch review -> resolved
2021-06-23 15:47:43vstinnersetmessages: + msg396428
2021-06-23 15:47:43vstinnersetmessages: + msg396427
2021-06-23 12:21:24vstinnersetpull_requests: + pull_request25454
2021-06-23 12:16:50vstinnersetpull_requests: + pull_request25453
2021-06-23 12:13:31vstinnersetmessages: + msg396414
2021-06-23 10:27:56vstinnersetmessages: + msg396405
2021-06-23 10:27:18vstinnersetmessages: + msg396404
2021-06-23 10:26:22vstinnersetpull_requests: + pull_request25449
2021-06-23 08:27:13kryhebsetfiles: + inittab-bug_no-threads.c

messages: + msg396385
2021-06-21 23:26:12vstinnersetmessages: + msg396300
2021-06-17 13:43:42petr.viktorinsetnosy: + vstinner
2021-06-17 10:21:12kryhebsetkeywords: + patch
stage: patch review
pull_requests: + pull_request25353
2021-06-17 10:14:12kryhebcreate