Title: Error in multiprocessing.Pool's initializer doesn't stop execution
Type: behavior Stage:
Components: Documentation, Library (Lib) Versions: Python 3.8
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: davin, docs@python, nemeskeyd, pitrou, terry.reedy
Priority: normal Keywords:

Created on 2021-02-23 15:55 by nemeskeyd, last changed 2021-02-27 03:38 by terry.reedy.

Messages (2)
msg387577 - (view) Author: Dávid Nemeskey (nemeskeyd) Date: 2021-02-23 15:55
There is an inconsistency in how multiprocessing.Pool handles exceptions thrown in the workers:
- exceptions raised by the mapped function stop execution right away
- exceptions raised in an initializer are ignored and the pool continues spawning new workers indefinitely, each of them failing.

I believe the behavior should be the same in both cases, and of the two, the first one is preferable (especially since that's what people are used to).

The documentation doesn't cover how exceptions are handled in pools, either.
msg387763 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-02-27 03:38
Can you add a minimal example with the ignore behavior?
Date User Action Args
2021-02-27 03:38:03terry.reedysetassignee: docs@python ->

messages: + msg387763
nosy: + terry.reedy, pitrou, davin
2021-02-23 15:55:10nemeskeydcreate