This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients JelleZijlstra, benjamin.peterson, corona10, remi.lapeyre, serhiy.storchaka, stutzbach, vstinner
Date 2020-06-01.12:53:35
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1591016017.05.0.331355757994.issue40826@roundup.psfhosted.org>
In-reply-to
Content
The following change modified PyOS_InterruptOccurred() to require the hold the GIL:

commit d83168854e19d0381fa57db25fca6c622917624f
Author: Victor Stinner <vstinner@python.org>
Date:   Fri Mar 20 14:50:35 2020 +0100

    bpo-40010: Optimize pending calls in multithreaded applications (GH-19091)
    
    If a thread different than the main thread schedules a pending call
    (Py_AddPendingCall()), the bytecode evaluation loop is no longer
    interrupted at each bytecode instruction to check for pending calls
    which cannot be executed. Only the main thread can execute pending
    calls.
    
    Previously, the bytecode evaluation loop was interrupted at each
    instruction until the main thread executes pending calls.
    
    * Add _Py_ThreadCanHandlePendingCalls() function.
    * SIGNAL_PENDING_CALLS() now only sets eval_breaker to 1 if the
      current thread can execute pending calls. Only the main thread can
      execute pending calls.

--

PyOS_InterruptOccurred() is part of the limited C API, but it's not even documented :-( So it's not easy to say if it's a backward incompatible change. At least, as the author of the change, I can say that the change was deliberate.

bpo-40010 rationale is that only the main thread of the main interpreter must call Python signal handlers. So when another thread or another interpreter (running the main thread) asks "should the Python signal handlers be executed", the answer is always "no", even if yes, Python got a signal (which requires to execute at least one Python signal handler).

The change is that PyOS_InterruptOccurred() now requires to hold the GIL to get current Python thread state: _PyThreadState_GET() returns NULL when the GIL is released.

Modifying PyOS_InterruptOccurred() to always return 0 if the GIL is released (if _PyThreadState_GET() returns NULL) is wrong: if the function is called by the main thread of the main interpreter, it must return 1.

--

By the way, I rewrote the C signal handler in Python 3.9 to not require to get the current Python thread state. In Python 3.8, it was modified and started to require the current Python thread if writing into the wakeup file descriptor failed to schedule a "pending call".

commit b54a99d6432de93de85be2b42a63774f8b4581a0
Author: Victor Stinner <vstinner@python.org>
Date:   Wed Apr 8 23:35:05 2020 +0200

    bpo-40082: trip_signal() uses the main interpreter (GH-19441)
    
    Fix the signal handler: it now always uses the main interpreter,
    rather than trying to get the current Python thread state.
    
    The following function now accepts an interpreter, instead of a
    Python thread state:
    
    * _PyEval_SignalReceived()
    * _Py_ThreadCanHandleSignals()
    * _PyEval_AddPendingCall()
    * COMPUTE_EVAL_BREAKER()
    * SET_GIL_DROP_REQUEST(), RESET_GIL_DROP_REQUEST()
    * SIGNAL_PENDING_CALLS(), UNSIGNAL_PENDING_CALLS()
    * SIGNAL_PENDING_SIGNALS(), UNSIGNAL_PENDING_SIGNALS()
    * SIGNAL_ASYNC_EXC(), UNSIGNAL_ASYNC_EXC()
    
    Py_AddPendingCall() now uses the main interpreter if it fails to the
    current Python thread state.
    
    Convert _PyThreadState_GET() and PyInterpreterState_GET_UNSAFE()
    macros to static inline functions.
History
Date User Action Args
2020-06-01 12:53:37vstinnersetrecipients: + vstinner, benjamin.peterson, stutzbach, serhiy.storchaka, JelleZijlstra, corona10, remi.lapeyre
2020-06-01 12:53:37vstinnersetmessageid: <1591016017.05.0.331355757994.issue40826@roundup.psfhosted.org>
2020-06-01 12:53:37vstinnerlinkissue40826 messages
2020-06-01 12:53:35vstinnercreate