classification
Title: Cannot clear signal handler set with loop.add_signal_handler in forked process with signal.signal
Type: behavior Stage:
Components: asyncio Versions: Python 3.10, Python 3.9, Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, selimb, yselivanov
Priority: normal Keywords:

Created on 2021-01-29 19:51 by selimb, last changed 2021-02-01 20:57 by selimb.

Files
File name Uploaded Description Edit
python-async-signal-bug.zip selimb, 2021-01-29 19:51 Experiments
Messages (2)
msg385928 - (view) Author: Selim Belhaouane (selimb) Date: 2021-01-29 19:51
Cannot clear signal handler set with loop.add_signal_handler in forked process with signal.signal

# Context

I'm running an async web server with uvicorn[1] and have background processes (using multiprocessing) doing CPU-bound work. uvicorn install signal handlers with loop.add*signal_handler, if available[2]. It also implements the common "attempt graceful shutdown on the first signal, and forcefully shutdown on the second (or more) signal" pattern. The problem I noticed was: when \_forking* at least one process, the server never gracefully shuts down, **even if I install new signal handlers in the subprocesses**.

# Problem

Signal handlers installed with loop.add*signal_handler cannot be cleared in forked processes with `signal.signal` (see Experiment 1), \_unless* using `signal.SIG_DFL` or `signal.SIG_IGN` in the forked processes (see Experiment 3).

For the record:

- When installing signal handlers in the parent process with signal.signal, this is not a problem (see Experiment 2).
- When using multiprocessing with the "spawn" method, this is not a problem. Unsurprising, since signal handlers are not inherited from the parent process when using "spawn" (see Experiment 4).
- When install signal handlers in the child proceess with loop.add_signal_handler, this is not a problem (see Experiment 5).

# Experiments

You'll find a minimal `exp.py` file in the attached archive. There's a few tweakable parameters at the top. I just run this with `python3.X exp.py`, wait a second or two, and hit Ctrl+C.
`results.txt` shows the results of a few experiments with different set of parameters.
Note that the point at which I hit Ctrl+C is indicated by the ♥ (heart) symbol, due to some terminal weirdness, although this is quite useful in this case!
The subsections below detail what happens when Ctrl+C is hit.
You guys probably know this already, but Ctrl+C basically sends SIGINT to all processes in the process tree.

## Experiment 1

- Installs a signal handler in the parent process with loop.add_signal_handler
- Starts 3 subprocesses with "fork"
- The subprocesses install a new signal handler with signal.signal

Outcome: both the parent handler and child handlers get called (3 calls to handle_sig_worker, 4 calls to handle_sig_main)

**Expected**: I would expect a single call to handle_sig_main, and 3 calls to handle_sig_worker, as in Experiment 2 (which uses signal.signal instead of loop.add_signal_handler) or Experiment 4 (which uses "spawn" instead of "fork") or Experiment 5 ()

## Experiment 2

As Experiment 1, but signal handlers are installed in the parent process with signal.signal

Outcome: the parent handler gets called once, and the child handlers get called 3 times

## Experiment 3

Same as Experiment 1, but this time using `signal.SIG_DFL` as the callback in the child processes.

Outcome: the child processes immediately terminate, thanks to SIG_DFL, and the parent handler gets called only once!

## Experiment 4

Same as Experiment 1, but this time using "spawn" as the multiprocessing start method.

Outcome: same as Experiment 2: parent handlers gets called once, child handlers get called 3 times

## Experiment 5

Same as Experiment 1, but this time installing signal handlers in the child processes with loop.add_signal_handler.

Outcome: Same as Experiment 2.

# Environment

I was able to replicate the issue with python 3.7, 3.8, 3.9 and 3.10 (didn't even try 3.6) on Ubuntu 20.04. I obtained python via `apt`.

[1]: https://www.uvicorn.org/
[2]: https://github.com/encode/uvicorn/blob/61a6cabb4580e1c923df396eac264803f599412c/uvicorn/server.py#L281
msg386112 - (view) Author: Selim Belhaouane (selimb) Date: 2021-02-01 20:57
Playing with this a bit more, I found a weird workaround. By adding the following to the top of worker_sync(), Experiment 1 produces the expected result.

def worker_sync():
    async def clear_asyncio_signal_handlers():
        loop = asyncio.get_event_loop()
        loop.add_signal_handler(signal.SIGINT, signal.SIG_IGN)
    asyncio.run(clear_asyncio_signal_handlers())

    if WORKER_SIGNAL_HANDLER is WorkerSignalHandler.NONE:
    [...]

FYI, this did *not* work (the outcome is still the same for Experiment 1):

def worker_sync():
    loop = asyncio.get_event_loop()
    loop.add_signal_handler(signal.SIGINT, signal.SIG_IGN)

    if WORKER_SIGNAL_HANDLER is WorkerSignalHandler.NONE:
    [...]


Maybe this all can be flagged as "as designed". However, perhaps it could be documented?
History
Date User Action Args
2021-02-01 20:57:38selimbsetmessages: + msg386112
2021-01-29 20:27:01selimbsettitle: Impossible to override signal handler set with add_signal_handler in forked process -> Cannot clear signal handler set with loop.add_signal_handler in forked process with signal.signal
2021-01-29 19:51:37selimbcreate