This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Partial moving of core objects to interpreter state is incorrect at best, unsafe at worse.
Type: behavior Stage: patch review
Components: Interpreter Core Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, eric.snow, erlendaasland, shihai1991, vstinner
Priority: normal Keywords: patch

Created on 2021-11-02 16:04 by Mark.Shannon, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 29366 merged Mark.Shannon, 2021-11-02 16:22
Messages (5)
msg405514 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-11-02 16:04
We currently have an unstable state in the VM where some core objects are static and some are per-interpreter.

For example, smalls ints are allocated per-interpreter, but many classes are allocated statically.
This means that if any int is reachable from a class, then references to per-interpreter objects can be left dangling, or be out of date.

E.g. consider this sequence:
1. Create an interpreter
2. Destroy it.
3. Create a new interpreter

`sys.float_info.n_unnamed_fields` causes a memory violation if the per-interpreter allocated 0 held by sys.float_info.n_unnamed_fields is freed.
If it is not freed, then `sys.float_info.n_unnamed_fields is 0` is False, meaning that there are two zeros present.

The above is just an example. Classes have many references to ints, floats, code  objects, etc. Any of those could have the same issue.

All objects that form the core object graph must either be entirely static, or entirely per-interpreter.

We cannot change from static to per-interpreter in a piecemeal fashion. It must be done all at once.
msg405628 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-11-03 16:22
New changeset acc89db9233abf4d903af9a7595a2ed7478fe7d3 by Mark Shannon in branch 'main':
bpo-45691: Make array of small ints static to fix use-after-free error. (GH-29366)
https://github.com/python/cpython/commit/acc89db9233abf4d903af9a7595a2ed7478fe7d3
msg406483 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-11-17 18:14
> many classes are allocated statically

Right. Changing that is an hard problem :-( See for example bpo-40601 "[C API] Hide static types from the limited C API".

I tried once to "free" / reset static types in Py_Finalize(), but it's hard to implement properly :-(
msg410811 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-01-17 17:24
Mark:
> `sys.float_info.n_unnamed_fields` causes a memory violation if the per-interpreter allocated 0 held by sys.float_info.n_unnamed_fields is freed.

I created bpo-46417 follow-up issue: "[subinterpreters] Clear static types in Py_Finalize()".
msg410812 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-01-17 17:31
> `sys.float_info.n_unnamed_fields` causes a memory violation if the per-interpreter allocated 0 held by sys.float_info.n_unnamed_fields is freed.
> If it is not freed, then `sys.float_info.n_unnamed_fields is 0` is False, meaning that there are two zeros present.

Python 3.9 and 3.10 are concerned by this issue: integer singletons are per-interpreter since Python 3.9.

Should Python 3.9 and 3.10 be fixed? "x is 0" is recommended and should be used. For example, the compiler emits a SyntaxWarning:

$ python3.9
>>> x=0
>>> x is 0
<stdin>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
True

I propose to only fix the main branch and so close the issue.
History
Date User Action Args
2022-04-11 14:59:52adminsetgithub: 89854
2022-01-18 12:45:01shihai1991setnosy: + shihai1991
2022-01-17 17:31:23vstinnersetmessages: + msg410812
2022-01-17 17:24:51vstinnersetmessages: + msg410811
2021-11-17 18:14:21vstinnersetmessages: + msg406483
2021-11-03 16:22:41Mark.Shannonsetmessages: + msg405628
2021-11-02 21:56:27erlendaaslandsetnosy: + erlendaasland
2021-11-02 16:22:07Mark.Shannonsetkeywords: + patch
stage: patch review
pull_requests: + pull_request27626
2021-11-02 16:04:49Mark.Shannoncreate