classification
Title: Remove _PyThreadState_GetFrame
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, brett.cannon, gregory.p.smith, vstinner
Priority: normal Keywords: patch

Created on 2020-03-12 17:40 by vstinner, last changed 2020-05-04 21:50 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19094 merged vstinner, 2020-03-20 16:26
Messages (7)
msg364031 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-12 17:40
Python has an internal function to get the frame of the PyThreadState:

/* hook for PyEval_GetFrame(), requested for Psyco */
#define _PyThreadState_GetFrame _PyRuntime.gilstate.getframe

It is used by the public function PyEval_GetFrame() for example.


The indirection was added in 2002 by:

commit 019a78e76d3542d4d56a08015e6980f8c8aeaba1
Author: Michael W. Hudson <mwh@python.net>
Date:   Fri Nov 8 12:53:11 2002 +0000

    Assorted patches from Armin Rigo:
    
    [ 617309 ] getframe hook (Psyco #1)
    [ 617311 ] Tiny profiling info (Psyco #2)
    [ 617312 ] debugger-controlled jumps (Psyco #3)
    
    These are forward ports from 2.2.2.


... but psyco is outdated for a very long time (superseded by PyPy which is no longer based on CPython). Is it time to drop _PyThreadState_GetFrame() (which became _PyRuntime.gilstate.getframe in the meanwhile)?

Or if we keep it, we should use it rather accessing directly PyThreadState.frame (read or write).

See also PEP 523 "Adding a frame evaluation API to CPython" and a recent discussion on this PEP: bpo-38500.
msg364675 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-20 14:52
I added PyThreadState_GetFrame() function in bpo-39947: commit fd1e1a18fa3befe5b6eeac32e0561e15c7e5164b.
msg364677 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-20 16:22
It looks safe to remove this feature.

I failed to find any recent user of _PyThreadState_GetFrame.

Moreover, _PyRuntime.getframe and _PyThreadState_GetFrame have been moved to the internal C API in Python 3.7. This C API was not accessible by third-party projects in Python 3.7. The internal C API can only be used by third-party projects since Python 3.8 and it's tricky to use it... on purpose. I don't recall any complain about _PyThreadState_GetFrame becoming inaccessible.

--

I searched for "_PyThreadState_GetFrame" in C code on GitHub. I only found copies of Python/pystate.c files in the first 10 pages of search results, with one exception. I found one file:

https://github.com/Mistobaan/pdb-clone/blob/517f6d19902b64395b4c7218cbbbecfa5a1de607/lib/pdb_clone/_pdbhandler-py27.c

It's an old (latest commit on Mar 31, 2015) debugger project written by Xavier de Gaye.

It seems like the following flavor is more recent (latest commit on Apr 20, 2019):

https://github.com/corpusops/pdbclone/

Note: the project contains 4 .c files, but only _pdbhandler-py27.c uses _PyThreadState_GetFrame: _pdbhandler-py3.c which is for Python 3 doesn't use _PyThreadState_GetFrame. But Python 2.7 reached its end of life, it's no longer supported.

Copy of the code:

    /* Disable the Python 2 restricted mode in the subinterpreter (see
     * PyEval_GetRestricted()) that prevents linecache to open the source
     * files and prevents attribute access. */
    saved_globals = mainstate->frame->f_globals;
    saved_locals = mainstate->frame->f_locals;
    saved_tstate_getframe = _PyThreadState_GetFrame;
    mainstate->frame->f_globals = globals;
    mainstate->frame->f_locals = locals;
    _PyThreadState_GetFrame = threadstate_getframe;
    pdbhandler_tstate = mainstate;

So _PyThreadState_GetFrame was used to "disable the Python 2 restricted mode", but this mode has been removed from Python 3.
msg364679 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-20 16:47
New changeset 6723e933c4d90a408cf3818362a0e4de6d84c932 by Victor Stinner in branch 'master':
bpo-39946: Remove _PyThreadState_GetFrame (GH-19094)
https://github.com/python/cpython/commit/6723e933c4d90a408cf3818362a0e4de6d84c932
msg364680 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-20 16:49
I removed the function.

FYI I asked Armin Rigo, the author of _PyThreadState_GetFrame, if PyPy uses it: no, it doesn't. Moreover, psyco project is outdated, only supports Python 2.6 and older, and is superseded by PyPy.
msg368084 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2020-05-04 21:49
Thanks!

We use this function internally in some VM traceback grabbing code but the best solution looks to just be for us to adopt the patch to 3.9 on our interpreter until we're running on 3.9.
msg368085 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-04 21:50
> We use this function internally in some VM traceback grabbing code

Would you mind to elaborate your use case?
History
Date User Action Args
2020-05-04 21:50:53vstinnersetmessages: + msg368085
2020-05-04 21:49:54gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg368084
2020-03-20 16:49:31vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg364680

stage: patch review -> resolved
2020-03-20 16:47:01vstinnersetmessages: + msg364679
2020-03-20 16:27:56vstinnersettitle: Is it time to remove _PyThreadState_GetFrame() hook? -> Remove _PyThreadState_GetFrame
2020-03-20 16:26:09vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request18454
2020-03-20 16:22:25vstinnersetmessages: + msg364677
2020-03-20 14:52:33vstinnersetmessages: + msg364675
2020-03-12 17:41:05vstinnersetnosy: + brett.cannon, Mark.Shannon
2020-03-12 17:40:48vstinnercreate