This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author tkc17
Recipients tkc17
Date 2021-06-19.19:49:48
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1624132188.66.0.0481912539758.issue44462@roundup.psfhosted.org>
In-reply-to
Content
This is similar to https://bugs.python.org/issue22393 but the child is killed in the Pool.terminate() using SIGTERM i.e., proper way.

Run the attached script that runs an indefinite method using multiprocessing.apply_async, and in the signal handler for SIGTERM does the pool clean up (close, terminate and join), on issuing a SIGTERM (`kill <parent PID>`), the cleanup works fine, but the parent process is stuck in get() as during termination the event is not set.

Able to repro with all 3.6/3.9/3.10.


See logs below
```
[DEBUG/MainProcess] created semlock with handle 140035884298240
[DEBUG/MainProcess] created semlock with handle 140035884294144
[DEBUG/MainProcess] created semlock with handle 140035884290048
[DEBUG/MainProcess] created semlock with handle 140035884285952
[DEBUG/MainProcess] created semlock with handle 140035884281856
[DEBUG/MainProcess] created semlock with handle 140035884277760
[DEBUG/MainProcess] added worker
[INFO/ForkPoolWorker-1] child process calling self.run()
[INFO/MainProcess] <function close_thread_pools at 0x7f5ca3a20b80>
[INFO/MainProcess] <function close_thread_pools at 0x7f5ca3a20b80>
[INFO/MainProcess] <function close_thread_pools at 0x7f5ca3a20b80>
[INFO/ForkPoolWorker-1] Signals for 28462
[INFO/ForkPoolWorker-1] Handlers.SIG_DFL
[INFO/ForkPoolWorker-1] <built-in function default_int_handler>
[INFO/ForkPoolWorker-1] Handlers.SIG_DFL
[INFO/MainProcess] 28461: Caught signal 15
[INFO/MainProcess] 28461: Closing pools
[INFO/MainProcess] 28461: Terminating pools
[DEBUG/MainProcess] terminating pool
[DEBUG/MainProcess] finalizing pool
[DEBUG/MainProcess] helping task handler/workers to finish
[DEBUG/MainProcess] removing tasks from inqueue until task handler finished
[DEBUG/MainProcess] joining worker handler
[DEBUG/MainProcess] worker handler exiting
[DEBUG/MainProcess] result handler found thread._state=TERMINATE
[DEBUG/MainProcess] ensuring that outqueue is not full
[DEBUG/MainProcess] result handler exiting: len(cache)=1, thread._state=TERMINATE
[DEBUG/MainProcess] task handler got sentinel
[DEBUG/MainProcess] task handler sending sentinel to result handler
[DEBUG/MainProcess] task handler sending sentinel to workers
[DEBUG/MainProcess] task handler exiting
[DEBUG/MainProcess] terminating workers
[DEBUG/MainProcess] joining task handler
[DEBUG/MainProcess] joining result handler
[DEBUG/MainProcess] joining pool workers
[DEBUG/MainProcess] cleaning up worker 28462
[INFO/MainProcess] 28461: Joining pools
[DEBUG/MainProcess] joining pool
[INFO/MainProcess] 28461: Closed pools
```

```
(gdb) py-bt
Traceback (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 312, in wait
    waiter.acquire()
  File "/usr/lib/python3.9/threading.py", line 574, in wait
    signaled = self._cond.wait(timeout)
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 762, in wait
    self._event.wait(timeout)
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 765, in get
    self.wait(timeout)
  File "/home/tatac/workspaces/gerrit/bwt-device-health-manager/mp_test.py", line 66, in <module>
    a.get()
(gdb) 
```
History
Date User Action Args
2021-06-19 19:49:48tkc17setrecipients: + tkc17
2021-06-19 19:49:48tkc17setmessageid: <1624132188.66.0.0481912539758.issue44462@roundup.psfhosted.org>
2021-06-19 19:49:48tkc17linkissue44462 messages
2021-06-19 19:49:48tkc17create