Title: ProcessPoolExecutor hangs on shutdown nowait with pickling failure
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.9
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: bquinlan, miss-islington, patbuxton, pitrou, tomMoral, vstinner
Priority: normal Keywords: patch

Created on 2019-12-20 06:52 by tomMoral, last changed 2020-04-27 22:08 by vstinner. This issue is now closed.

File name Uploaded Description Edit tomMoral, 2019-12-20 06:52
Pull Requests
URL Status Linked Edit
PR 17670 merged tomMoral, 2019-12-20 06:54
Messages (5)
msg358697 - (view) Author: Thomas Moreau (tomMoral) * Date: 2019-12-20 06:52
The attached scripts hangs on python3.7+.
This is due to the fact that the main process closes the communication channels directly while the queue_management_thread might still use them.

To prevent that, all the closing should be handled by the queue_management_thread.
msg360051 - (view) Author: Patrick Buxton (patbuxton) * Date: 2020-01-15 13:00
Added core developer to get some feedback
msg362088 - (view) Author: miss-islington (miss-islington) Date: 2020-02-16 18:09
New changeset a5cbab552d294d99fde864306632d7e511a75d3c by Thomas Moreau in branch 'master':
bpo-39104: Fix hanging ProcessPoolExecutor on shutdown nowait with pickling failure (GH-17670)
msg362089 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2020-02-16 18:10
Thank you Thomas. Ii'd rather not backport delicate changes, so closing now.
msg367466 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-27 22:08
> bpo-39104: Fix hanging ProcessPoolExecutor on shutdown nowait with pickling failure (GH-17670)

ProcessPoolSpawnProcessPoolExecutorTest.test_killed_child() of test_concurrent_futures started to fail randomly since this change: see bpo-39995.
Date User Action Args
2020-04-27 22:08:26vstinnersetnosy: + vstinner
messages: + msg367466
2020-02-16 18:10:37pitrousetstatus: open -> closed
versions: - Python 3.7, Python 3.8
type: behavior
messages: + msg362089

resolution: fixed
stage: patch review -> resolved
2020-02-16 18:09:29miss-islingtonsetnosy: + miss-islington
messages: + msg362088
2020-01-27 14:20:26patbuxtonsetnosy: + bquinlan
2020-01-15 13:00:57patbuxtonsetnosy: + pitrou, patbuxton
messages: + msg360051
2019-12-20 06:54:22tomMoralsetkeywords: + patch
stage: patch review
pull_requests: + pull_request17134
2019-12-20 06:52:22tomMoralcreate