classification
Title: _PyRuntime_Initialize() called after Py_Finalize() does nothing
Type: Stage: resolved
Components: Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: vstinner
Priority: normal Keywords: patch

Created on 2018-12-03 15:13 by vstinner, last changed 2019-03-19 23:03 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 12443 merged vstinner, 2019-03-19 16:49
Messages (3)
msg330946 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-03 15:13
When Python is embedded, it should be possible to call the following Python function multiple times:

void func(void)
{
  Py_Initialize();
  /* do something in Python */
  Py_Finalize();
}

Py_Finalize() ends by calling _PyRuntime_Finalize().

Problem: when Py_Initialize() is called the second time, _PyRuntime_Initialize() does nothing:

_PyInitError
_PyRuntime_Initialize(void)
{
    /* XXX We only initialize once in the process, which aligns with
       the static initialization of the former globals now found in
       _PyRuntime.  However, _PyRuntime *should* be initialized with
       every Py_Initialize() call, but doing so breaks the runtime.
       This is because the runtime state is not properly finalized
       currently. */
    static int initialized = 0;
    if (initialized) {
        return _Py_INIT_OK();
    }
    initialized = 1;

    return _PyRuntimeState_Init(&_PyRuntime);
}

For example, Py_Finalize() clears runtime->interpreters.mutex and runtime->xidregistry.mutex, whereas mutexes are still needed the second time func() is called.

There is currently a *workaround*:

_PyInitError
_PyInterpreterState_Enable(_PyRuntimeState *runtime)
{
    ...
    if (runtime->interpreters.mutex == NULL) {
        ...
        runtime->interpreters.mutex = PyThread_allocate_lock();
        ...
    }
    ...
}

I would prefer that _PyRuntime_Initialize() calls _PyRuntimeState_Init() each time, and that _PyRuntimeState_Init() does nothing at the following call (except after Py_Finalize?).

Note: _PyRuntimeState_Fini() doesn't free runtime->xidregistry.mutex currently.
msg338386 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-19 16:50
PR 12443 fix the issue.
msg338415 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-19 23:03
New changeset fd23cfa464ab93273370475900819c1ea37c852f by Victor Stinner in branch 'master':
bpo-35388: Fix _PyRuntime_Finalize() (GH-12443)
https://github.com/python/cpython/commit/fd23cfa464ab93273370475900819c1ea37c852f
History
Date User Action Args
2019-03-19 23:03:17vstinnersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-03-19 23:03:05vstinnersetmessages: + msg338415
2019-03-19 16:50:35vstinnersetmessages: + msg338386
2019-03-19 16:49:39vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request12398
2018-12-03 15:13:10vstinnercreate