classification
Title: Deadlocks in `concurrent.futures.ProcessPoolExecutor` with pickling error
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: pitrou, tomMoral
Priority: normal Keywords: patch

Created on 2017-10-05 07:41 by tomMoral, last changed 2018-01-05 12:33 by pitrou. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 3895 merged tomMoral, 2017-10-05 07:41
PR 4256 open tomMoral, 2017-11-03 15:14
Messages (3)
msg303745 - (view) Author: Thomas Moreau (tomMoral) * Date: 2017-10-05 07:41
When using `concurrent.futures.ProcessPoolExecutor` with objects that are not picklable or unpicklable, several situations results in a deadlock, with the interpreter freezed.

This is the case for different scenario, for instance these three : https://gist.github.com/tomMoral/cc27a938d669edcf0286c57516942369

The different pickling/unpickling error and their effect should be tested in `test_concurrent_futures.py`
msg309497 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-01-05 10:15
New changeset 94459fd7dc25ce19096f2080eb7339497d319eb0 by Antoine Pitrou (Thomas Moreau) in branch 'master':
bpo-31699 Deadlocks in `concurrent.futures.ProcessPoolExecutor` with pickling error (#3895)
https://github.com/python/cpython/commit/94459fd7dc25ce19096f2080eb7339497d319eb0
msg309504 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-01-05 12:33
This non-trivial issue has been fixed now, thanks to Thomas' patch.  Thank you Thomas!
History
Date User Action Args
2018-01-05 12:33:10pitrousetstatus: open -> closed
type: behavior
messages: + msg309504

resolution: fixed
stage: patch review -> resolved
2018-01-05 10:15:57pitrousetnosy: + pitrou
messages: + msg309497
2017-11-03 15:14:48tomMoralsetkeywords: + patch
stage: patch review
pull_requests: + pull_request4218
2017-10-05 07:41:45tomMoralcreate