classification
Title: Deadlocks in `concurrent.futures.ProcessPoolExecutor`
Type: behavior Stage: resolved
Components: Library (Lib) Versions:
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Deadlocks in `concurrent.futures.ProcessPoolExecutor` with pickling error
View: 31699
Assigned To: Nosy List: bquinlan, davin, pitrou, tomMoral
Priority: normal Keywords:

Created on 2017-04-06 12:41 by tomMoral, last changed 2019-05-30 12:33 by pitrou. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 1013 closed tomMoral, 2017-04-06 12:41
Messages (4)
msg291224 - (view) Author: Thomas Moreau (tomMoral) * Date: 2017-04-06 12:41
The design of ProcessPoolExecutor contains some possible race conditions that may freeze the interpreter due to deadlocks. This is notably the case  with pickling and unpickling errors for a submitted job and returned results. This makes it hard to reuse a launched executor.

We propose in the joint PR to fix some of those situations to make the ProcessPoolExecutor more robust to failure in the different threads and worker.
msg341898 - (view) Author: Brian Quinlan (bquinlan) * (Python committer) Date: 2019-05-08 16:03
Was this fixed by https://github.com/python/cpython/pull/3895 ?
msg343960 - (view) Author: Thomas Moreau (tomMoral) * Date: 2019-05-30 09:41
The deadlocks I refer to in this issue are fixed by the PR #3895.
Subsequent failures (like the fact that the Executor is set in a broken state when there is an unpickling error) are tracked in other issues so I think it is safe to close this one.
msg343973 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2019-05-30 12:33
Ok, closing as duplicate then.
History
Date User Action Args
2019-05-30 12:33:22pitrousetstatus: open -> closed
resolution: duplicate
messages: + msg343973

superseder: Deadlocks in `concurrent.futures.ProcessPoolExecutor` with pickling error
stage: patch review -> resolved
2019-05-30 09:41:34tomMoralsetmessages: + msg343960
2019-05-08 16:03:43bquinlansetmessages: + msg341898
2017-10-23 17:03:01berker.peksaglinkissue30549 superseder
2017-04-08 11:57:53pitrousetnosy: + bquinlan, pitrou, davin

stage: patch review
2017-04-06 12:41:11tomMoralcreate