classification
Title: [subinterpreters] Add --experimental-isolated-subinterpreters build option
Type: Stage: patch review
Components: Build, Subinterpreters Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: corona10, eric.snow, vstinner
Priority: normal Keywords: patch

Created on 2020-05-05 13:13 by vstinner, last changed 2020-06-05 01:16 by vstinner.

Pull Requests
URL Status Linked Edit
PR 19926 merged vstinner, 2020-05-05 13:39
PR 20228 merged vstinner, 2020-05-19 14:03
Messages (6)
msg368141 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-05 13:13
Modifying Python internals to have one GIL per interpreter (bpo-40512) is a large project which requires to modify many small things (again, see bpo-40512).

I propose to add a temporary build --experimental-isolated-subinterpreters build option to configure for developers who want to hack on isolated subinterpreters. The intent is to speedup bpo-40512 development by making some practice compromises, to have more time to properly design the real fixes.

For example, tuple, dict and frame have free lists which are shared by subinterpreters. A practical solution is to simply disable them at build time to avoid the need to have per-interpreter free lists.

Another example is pymalloc which is shared by all subinterpreters and rely on the unique global interpreter lock (GIL) to protect its internal states. A practical solution is to disable it and force the usage of libc malloc() function instead.

Some compromosises cannot be the default since they have a significant negative impact on performances. So I propose to add a temporary build option until all these small issues will be fixed.
msg368156 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-05 14:41
New changeset c5fa364f4ea836f25dd07cfb328152d40a568371 by Victor Stinner in branch 'master':
bpo-40514: Add --with-experimental-isolated-subinterpreters (GH-19926)
https://github.com/python/cpython/commit/c5fa364f4ea836f25dd07cfb328152d40a568371
msg368159 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-05 14:46
--with-experimental-isolated-subinterpreters option name is very long on purpose: to advertize that you must not use it, unless you fully understand its purpose :-)

I didn't document the build option in Misc/SpecialBuilds.txt or Doc/whatsnew/3.9.rst on purpose: you should not use this special build :-) Basically, this option is only for Eric Snow and me :-D
msg368172 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2020-05-05 15:33
It would probably make sense to remove the build option in the 3.9 release.  We can leave it in master, but remove it in the 3.9 branch once it has been created.
msg369401 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-19 22:27
New changeset 9512ad74b0fcaff023c1ade75313dc8e249aef78 by Victor Stinner in branch '3.9':
[3.9] bpo-40514: Remove --with-experimental-isolated-subinterpreters in 3.9 (GH-20228)
https://github.com/python/cpython/commit/9512ad74b0fcaff023c1ade75313dc8e249aef78
msg370745 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-05 01:16
I started to *remove* a few"#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS":

* commit 88ec9190105c9b03f49aaef601ce02b242a75273: Make list free list per-interpreter
* commit 3744ed2c9c0b3905947602fc375de49533790cb9: Make frame free list per-interpreter
* PR 20645 (under review): Make dict free lists per-interpreter
History
Date User Action Args
2020-06-05 01:16:12vstinnersetversions: + Python 3.10, - Python 3.9
2020-06-05 01:16:01vstinnersetmessages: + msg370745
2020-05-19 22:27:53vstinnersetmessages: + msg369401
2020-05-19 14:03:36vstinnersetpull_requests: + pull_request19519
2020-05-15 00:35:50vstinnersetcomponents: + Subinterpreters
title: Add --experimental-isolated-subinterpreters build option -> [subinterpreters] Add --experimental-isolated-subinterpreters build option
2020-05-05 15:33:32eric.snowsetmessages: + msg368172
2020-05-05 15:12:33vstinnersetnosy: + eric.snow
2020-05-05 14:46:13vstinnersetmessages: + msg368159
2020-05-05 14:41:17vstinnersetmessages: + msg368156
2020-05-05 14:19:16corona10setnosy: + corona10
2020-05-05 13:39:51vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request19241
2020-05-05 13:13:51vstinnercreate