This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Avoid allocating when exiting frame; it may be unsafe.
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.11
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Mark.Shannon Nosy List: Mark.Shannon, pablogsal, vstinner
Priority: normal Keywords: 3.11regression, patch

Created on 2021-11-11 15:04 by Mark.Shannon, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 29729 merged Mark.Shannon, 2021-11-23 15:20
PR 31874 merged vstinner, 2022-03-14 16:56
Messages (6)
msg406163 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-11-11 15:04
We exiting a frame (returning from a Python function) we have to release the stack allocated frame. If a heap-allocated frame object exists, we need to copy the contents of the (stack) frame into the frame object.
However, this involves allocating memory for the copy. Allocating memory can invoke GC, causing arbitrary code to be run, or the allocation can fail. Either leaves us in a precarious state, which may be unsafe.

I haven't been able to produce a crash, but I'm not sure that there isn't a potential crash lurking there either.

The fix is fairly simple. Allocate space for the copy of the frame at the end of the frame object. Then we need to copy the data, space will have already been allocated, and nothing can fail.

Since, in theory, heap-allocated frames are relatively rare, the extra memory used won't be an issue.
msg406165 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-11-11 16:38
> Allocating memory can invoke GC

Technically, only allocating objects can trigger GC (specifically initializing them). All the malloc APIs don't currently call the GC by themselves.
msg406166 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-11-11 16:56
Ok, so it is not as bad as I thought.

A failed allocation might leave us with an invalid frameobject, though. So it is still worth fixing.
msg407271 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-11-29 12:35
New changeset 60929576e40038ec71d896230f69e4411c82be4b by Mark Shannon in branch 'main':
bpo-45786: Allocate space for frame in frame object. (GH-29729)
msg410506 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2022-01-13 18:35
With both frame objects and generators contain space for the frame, so no allocation occurs on exit.
msg415480 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-18 10:17
New changeset 2217462bda1865a047d358306088682ee6a091ed by Victor Stinner in branch 'main':
bpo-45786: Remove _PyFrame_Fini() and _PyFrame_DebugMallocStats() (GH-31874)
Date User Action Args
2022-04-11 14:59:52adminsetgithub: 89944
2022-03-18 10:17:20vstinnersetmessages: + msg415480
2022-03-14 16:56:41vstinnersetnosy: + vstinner

pull_requests: + pull_request29972
2022-01-13 18:35:35Mark.Shannonsetstatus: open -> closed
resolution: fixed
messages: + msg410506

stage: patch review -> resolved
2021-11-29 12:35:08Mark.Shannonsetmessages: + msg407271
2021-11-23 15:20:00Mark.Shannonsetkeywords: + patch
stage: patch review
pull_requests: + pull_request27966
2021-11-11 16:56:58Mark.Shannonsetmessages: + msg406166
2021-11-11 16:38:44pablogsalsetmessages: + msg406165
2021-11-11 15:04:09Mark.Shannoncreate