classification
Title: [C API] Enforce usage of PyFrame_GetBack()
Type: Stage:
Components: C API Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, silverjubleegaming, vstinner
Priority: normal Keywords:

Created on 2022-01-12 15:33 by vstinner, last changed 2022-01-12 17:02 by vstinner.

Messages (5)
msg410403 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-01-12 15:33
C extensions written for Python 3.10 and older which access directly to the PyFrameObject.f_back member build successfully on Python 3.11, but they can fail because f_back must not be read directly. f_back can be NULL even if the frame has an outer frame.

PyFrameObject*
PyFrame_GetBack(PyFrameObject *frame)
{
    assert(frame != NULL);
    PyFrameObject *back = frame->f_back;
    if (back == NULL && frame->f_frame->previous != NULL) {
        back = _PyFrame_GetFrameObject(frame->f_frame->previous);
    }
    Py_XINCREF(back);
    return back;
}

I suggest to remove or "hide" this member from the structure. For example, rename "f_back" to "_f_back" to advice developers to not access it directly.

See also bpo-46355: Document PyFrameObject and PyThreadState changes.
msg410405 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-01-12 15:39
It's unclear to me if the ability to read directly the PyFrameObject.f_lineno member is a deliberate use case, or if we should enforce the usage of PyFrame_GetLineNumber().
msg410406 - (view) Author: Silver Jublee Gaming (silverjubleegaming) Date: 2022-01-12 15:42
This must be normal
msg410408 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2022-01-12 16:14
f_lineno is not part of the API. No field in PyFrameObject is.

I don't know how we can stop people from using them though.

If they don't know better than pulling data out of undocumented internal struct, then I'm not sure an underscore prefix is going to help.
It won't do any harm, though.
msg410415 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-01-12 17:02
> I don't know how we can stop people from using them though.

The first option is to promote helper functions to abstract access to the PyFrameObject structure.

The second option is to make the whole structure opaque and enforce the usage of helper functions. Before being able to do that, we need to prepare Cython and a few more popular C extensions for that. Right now, Cython still has a lot of code accessing directly PyFrameObject members.

Well, we can discuss that in bpo-40421 ;-)


> If they don't know better than pulling data out of undocumented internal struct, then I'm not sure an underscore prefix is going to help. It won't do any harm, though.

I propose to rename f_back to trigger a build error for existing C extensions which access it directly. So developers have to decide how to handle the change: access the renamed member, or read the doc and use a better way, call PyFrame_GetBack() :-)

I dislike that this incompatible change is "silent" and that developers may only notice the change in production, when it's too late.

Or maybe most C extensions have a a good test suite and will notice the change before it's too late.
History
Date User Action Args
2022-01-12 17:02:21vstinnersetmessages: + msg410415
2022-01-12 16:14:55Mark.Shannonsetmessages: + msg410408
2022-01-12 15:42:24silverjubleegamingsetnosy: + silverjubleegaming
messages: + msg410406
2022-01-12 15:39:13vstinnersetmessages: + msg410405
2022-01-12 15:33:01vstinnercreate