I partially reimplemented commit ef4ac967e2f3a9a18330cc6abe14adb4bc3d0465 in the following issues:

* bpo-39984: Move some ceval fields from _PyRuntime.ceval to PyInterpreterState.ceval
* bpo-40010: Inefficient signal handling in multithreaded applications
* bpo-39877: Daemon thread is crashing in PyEval_RestoreThread() while the main thread is exiting the process
* bpo-37127: Handling pending calls during runtime finalization may cause problems.

I cannot give a list of commits, I made too many of them :-D


The most important change is:

commit 50e6e991781db761c496561a995541ca8d83ff87
Author: Victor Stinner <vstinner@python.org>
Date:   Thu Mar 19 02:41:21 2020 +0100

    bpo-39984: Move pending calls to PyInterpreterState (GH-19066)
    
    If Py_AddPendingCall() is called in a subinterpreter, the function is
    now scheduled to be called from the subinterpreter, rather than being
    called from the main interpreter.
    
    Each subinterpreter now has its own list of scheduled calls.
    
    * Move pending and eval_breaker fields from _PyRuntimeState.ceval
      to PyInterpreterState.ceval.
    * new_interpreter() now calls _PyEval_InitThreads() to create
      pending calls lock.
    * Fix Py_AddPendingCall() for subinterpreters. It now calls
      _PyThreadState_GET() which works in a subinterpreter if the
      caller holds the GIL, and only falls back on
      PyGILState_GetThisThreadState() if _PyThreadState_GET()
      returns NULL.


My plan is now to fix pending calls in subinterpreters. Currently, they are only executed in the "main thread"... _PyRuntimeState.main_thread must be moved to PyInterpreterState, as it was done in commit ef4ac967e2f3a9a18330cc6abe14adb4bc3d0465. 

This issue shows that it's dangerous to change too many things at once. Python internals were still very fragile: bpo-39877 and and bpo-37127 are good examples of that. I'm trying to move *very slowly*. Move pieces one by one.

I added _Py_ThreadCanHandlePendingCalls() function in commit d83168854e19d0381fa57db25fca6c622917624f (bpo-40010): it should ease moving _PyRuntimeState.main_thread to PyInterpreterState. I also added _Py_ThreadCanHandleSignals() in a previous commit, so it's easier to change which thread is allowed or not to handle signals and pending calls.

I also plan to revisit (removal) pending_calls.finalizing added by commit 842a2f07f2f08a935ef470bfdaeef40f87490cfc (bpo-33608). I plan to work on that in bpo-37127.