I've been struggling to get signal.pthread_sigmask to do what I expected it to do from the documentation. Having looked at the core python code handling signals I now (think?!) I understand what is happening. It might be possible for python to improve the behaviour, or it might just be something to document, I'm not sure but I though I'd mention it.

I'd added pthread_sigmask(SIG_BLOCK, (SIGTERM,)) and pthread_sigmask(SIG_UNBLOCK, (SIGTERM,)) calls around a critical section I wanted to protect from the SIGTERM signal. I was still seeing SIGTERM inside that section. Using SIGMASK to restore the mask instead of SIG_UNBLOCK behaves the same.

What I hadn't realised is that firstly python defers signals to a convenient point and secondly that signals are processed in the main thread regardless of the thread they arrived in. This means that I can see SIGTERM arrive in my critical section as one of my other threads created in the background by the core python libs helpfully handles it.  This makes SIG_BLOCK rather ineffective in any threaded code.

To work around it, I can add my own handlers and have them track whether a signal arrived, then handle any signals after my critical section by re-raising them. It is possible python itself could defer processing signals masked with SIG_BLOCK until they're unblocked. Alternatively, a note in the documentation warning of the pitfalls here might be helpful to save someone else from wondering what is going on!

The irony... Documenting the caveat at least seems useful. Your workaround sounds reasonable.

I don't love the idea of implementing our own mask blocked/unblocked state check, though it probably wouldn't be very complicated. Might be interesting.

Another trick that'd "work" so long as people don't have multiple SIG_BLOCK calls (not something to depend on) is to force a check of our interpreter signal flags right before signal.pthread_sigmask returns to the Python caller.

The "trick" wouldn't be too useful though as this API can't block and the signal flag needs to be processed on the main thread. So I guess documentation it is.

The way I think of this is that the signal.pthread_sigmask API is pretty low level. After that API is called, no more signals will _reach the process_, but the interpreter may process some that happened beforehand. So installing a handler is indeed appropriate.

Also, it is entirely possible for C extension modules or code embedding Python to call pthread_sigmask in its own threads outside of the Python runtimes knowledge. So we us tracking what signals are blocked on our own may not be accurate.  We could instead change the eval loop signal processing code to call `pthread_sigmask(SIG_UNBLOCK, NULL /* set */, &oldset);` to retrieve the processes current mask to decide if the flagged signals should be processed by Python or not.

BUT... I can imagine complex race cases where that'd surprise people who are chaining multiple signal handlers such as one from outside of Python that saved the Python handler and calls it afterwards.  Their C/process-level handler would be called, would chain to Python's "record that signal X happened and set the bit" handler, but Python wouldn't then be guaranteed to call the Python handler code if the sigmask changed before the eval loop did its pending signal check.

So I'm still inclined to keep this simple and stick with just documenting best practices for now.  An implementation of masking from the python eval handler may need to be something conditionally controllable for differing application situations if added.

I think the python code implementing pthread_sigmask already does trigger interrupts if any have been queued before the function returns from blocking or unblocking.

The key subtlety which I initially missed is that if you have another thread in your python script, any interrupt it receives can be raised in the main thread whilst you're in the SIGBLOCK section. This obviously isn't what you expect at all as those interrupts are supposed to be blocked! It isn't really practical to try and SIGBLOCK on all your individual threads.

What I'd wondered is what you mention, specifically checking if a signal is masked in the python signal raising code with something like the "pthread_sigmask(SIG_UNBLOCK, NULL /* set */, &oldset)" before it raises it and if there is blocked, just leave it queued. The current code would trigger the interrupts when it was unmasked. This would effectively only apply on the main thread where all the signals/interrupts are raised. 

This would certainly give the behaviour that would be expected from the calls and save everyone implementing the workarounds as I have. Due to the threads issue, I'm not sure SIGBLOCK is actually useful in the real world with the current implementation unfortunately.

Equally, if that isn't an acceptable fix, documenting it would definitely be good too.