This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: concurrent.futures.InterpreterPoolExecutor
Type: enhancement Stage: needs patch
Components: Extension Modules Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: crusaderky, davin, eric.snow, jakirkham
Priority: normal Keywords:

Created on 2019-06-15 13:15 by crusaderky, last changed 2022-04-11 14:59 by admin.

Messages (3)
msg345681 - (view) Author: Guido Imperiale (crusaderky) * Date: 2019-06-15 13:15
As one of the logical consequences to PEP 554, it would be neat to have a concurrent.futures.InterpreterPoolExecutor.

I wrote the initial code at https://github.com/crusaderky/subinterpreters_tests - currently missing unit tests and pickle5 buffers support.

If everybody is happy with the design, I'll start working on a PR as soon as the GIL becomes per-interpreter (it's kinda pointless before that).
msg346225 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2019-06-21 15:27
FWIW, performance benefits when subinterpreters stop sharing the GIL are not the only benefit.  In fact, PEP 554 is specifically written to avoid that consideration, focusing on the benefits of the concurrency model (i.e. CSP).  So I wouldn't call this pointless even without per-interpreter GIL. :)

(Of course, this *would* be blocked on acceptance and implementation of PEP 554.)
msg346226 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2019-06-21 15:28
@Davin, we've spoken before about something similar for multiprocessing, IIRC. :)
History
Date User Action Args
2022-04-11 14:59:16adminsetgithub: 81474
2020-08-05 21:42:09jakirkhamsetnosy: + jakirkham
2019-06-21 15:28:08eric.snowsetnosy: + davin
messages: + msg346226
2019-06-21 15:27:37eric.snowsetmessages: + msg346225
stage: needs patch
2019-06-15 13:15:17crusaderkycreate