Title: Feature request: maxtasksperchild for ProcessPoolExecutor
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.11
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: cool-RR, loganasherjones, pitrou
Priority: normal Keywords: patch

Created on 2021-07-24 12:14 by cool-RR, last changed 2021-07-26 17:39 by loganasherjones.

Pull Requests
URL Status Linked Edit
PR 27373 open loganasherjones, 2021-07-26 17:39
Messages (3)
msg398143 - (view) Author: Ram Rachum (cool-RR) * Date: 2021-07-24 12:14
I love `concurrent.futures`, and I'd like to use it wherever I can. There's a feature in `multiprocessing.Pool` that I wish would also be available in `ProcessPoolExecutor`: The `maxtasksperchild` argument.

Documentation: "maxtasksperchild is the number of tasks a worker process can complete before it will exit and be replaced with a fresh worker process, to enable unused resources to be freed. The default maxtasksperchild is None, which means worker processes will live as long as the pool."

I want to be able to set it to 1, so each process will only execute one task and then be replaced with a fresh process.
msg398240 - (view) Author: Logan Jones (loganasherjones) * Date: 2021-07-26 16:05
I think I have a solution for this, but I'm pretty new to contributing. Still writing up some tests.
msg398243 - (view) Author: Ram Rachum (cool-RR) * Date: 2021-07-26 16:12
Awesome. You can link to experimental code here. Even if it were not accepted to Python, it could be useful for me and for other people who might see this issue.
Date User Action Args
2021-07-26 17:39:39loganasherjonessetkeywords: + patch
stage: patch review
pull_requests: + pull_request25911
2021-07-26 16:12:19cool-RRsetmessages: + msg398243
2021-07-26 16:05:54loganasherjonessetmessages: + msg398240
2021-07-25 12:57:33loganasherjonessetnosy: + loganasherjones
2021-07-24 12:14:41cool-RRcreate