The long-term goal of the PEP 554 is to run two Python interpreters in parallel. To achieve this goal, no object must be shared between two interpreters. See for example my article "Pass the Python thread state explicitly" which gives a longer rationale:
https://vstinner.github.io/cpython-pass-tstate.html

In bpo-38858, I modified Objects/longobject.c to have per-interpreter small integer singletons: commit 630c8df5cf126594f8c1c4579c1888ca80a29d59.

This issue is about other singletons like None or Py_True which are currently shared between two interpreters.

I propose to add new functions. Example for None:

* Py_GetNone(): return a *borrowed* reference to the None singleton (similar to existing Py_None macro)
* Py_GetNoneRef(): return a *strong* reference to the None singleton (similar to "Py_INCREF(Py_None); return Py_None;" and Py_RETURN_NONE macro)

And add PyInterpreterState.none field: strong reference to the per-interpreter None object.


We should do that for each singletons:

* None (Py_None)
* True (Py_True)
* False (Py_False)
* Ellipsis (Py_Ellipsis)


GIL issue
=========

Py_GetNone() would look like:

PyObject* Py_GetNone(void)
{ return _PyThreadState_GET()->interp->none; }

Problem: _PyThreadState_GET() returns NULL if the caller function doesn't hold the GIL.

Using the Python C API when the GIL is not held is a violation of the API: it is not supported. But it worked previously.

One solution is to fail with an assertion error (abort the process) in debug mode, and let Python crash in release mode.

Another option is to only fail with an assertion error in debug mode in Python 3.9. In Python 3.9, Py_GetNone() would use PyGILState_GetThisThreadState() function which works even when the GIL is released. In Python 3.10, we would switch to _PyThreadState_GET() and so crash in release mode.

One concrete example of such issue can be found in the multiprocessing C code, in semlock_acquire():

            Py_BEGIN_ALLOW_THREADS
            if (timeout_obj == Py_None) {
                res = sem_wait(self->handle);
            }
            else {
                res = sem_timedwait(self->handle, &deadline);
            }
            Py_END_ALLOW_THREADS

Py_None is accessed when the GIL is released.