➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: multiprocessing.Pool - crash in subprocess causes deadlock in parent
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.10, Python 3.9, Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: jkp
Priority: normal Keywords:

Created on 2021-03-09 18:15 by jkp, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
test.py jkp, 2021-03-09 18:15
Messages (2)
msg388371 - (view) Author: Jamie Kirkpatrick (jkp) Date: 2021-03-09 18:15 
When using multiprocessing.Pool.apply[_async] a crash in the subprocess that is assigned the work item results in a deadlock in the parent process.

The parent process remains blissfully unaware of the crash in the subprocess and waits for a result forever. The parent process treats this as normal since the thread running _maintain_pool handles dead processes and repopulates the pool with a replacement subprocess.

See the test-case attached. Its not clear how this case should be handled but it can be very hard to trace issues in an application where this condition arises since all information about the crashing subprocess is lost (even with debug logging for the multiprocessing module enabled).
msg388381 - (view) Author: Jamie Kirkpatrick (jkp) Date: 2021-03-09 20:36 
More reading around this issue and I stumbled on an existing issue which this is a dup of so it can be closed.

https://bugs.python.org/issue22393
History
Date User Action Args
2022-04-11 14:59:42adminsetgithub: 87615
2021-03-09 20:36:37jkpsetstatus: open -> closed

messages: + msg388381
stage: resolved
2021-03-09 18:15:25jkpcreate