Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pass explicitly tstate to function calls #82825

Closed
vstinner opened this issue Oct 30, 2019 · 24 comments
Closed

Pass explicitly tstate to function calls #82825

vstinner opened this issue Oct 30, 2019 · 24 comments
Labels
3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@vstinner
Copy link
Member

BPO 38644
Nosy @rhettinger, @vstinner, @encukou
PRs
  • bpo-38644: Pass tstate to Py_EnterRecursiveCall() #16997
  • bpo-38644: Add Py_EnterRecursiveCall() to the limited API #17046
  • bpo-38644: Pass tstate to _Py_CheckFunctionResult() #17050
  • bpo-38644: Add _PyObject_VectorcallTstate() #17052
  • bpo-38644: Add _PyObject_Call() #17089
  • bpo-38644: Add _PyEval_EvalFrame() with tstate #17131
  • bpo-38644: Add _PyEval_EvalCode() #17183
  • bpo-38644: Cleanup ceval.h #17185
  • bpo-38644: Pass tstate to _Py_FinishPendingCalls() #17990
  • bpo-38644: Pass tstate in ceval.c #18222
  • bpo-38644: Add Py_EnterRecursiveCall() to python3.def #18399
  • bpo-38644: Rephrase What's New entry #18461
  • [WIP] bpo-39984: Add PyInterpreterState.ceval #19034
  • bpo-38644: Add _PySys_Audit() which takes tstate #19180
  • bpo-38644: Make tstate more explicit inside pystate.c #19182
  • bpo-38644: Use _PySys_Audit(): pass tstate explicitly #19183
  • bpo-38644: Pass tstate explicitly in signalmodule.c #19184
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-04-14.00:37:59.857>
created_at = <Date 2019-10-30.14:31:24.358>
labels = ['interpreter-core', '3.9']
title = 'Pass explicitly tstate to function calls'
updated_at = <Date 2021-09-07.10:32:41.166>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2021-09-07.10:32:41.166>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-04-14.00:37:59.857>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2019-10-30.14:31:24.358>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 38644
keywords = ['patch']
message_count = 24.0
messages = ['355716', '355717', '355974', '355984', '355987', '356234', '356495', '356597', '356601', '356732', '356734', '356878', '359916', '359920', '360835', '361558', '361559', '361825', '365104', '365112', '365113', '365152', '401213', '401225']
nosy_count = 3.0
nosy_names = ['rhettinger', 'vstinner', 'petr.viktorin']
pr_nums = ['16997', '17046', '17050', '17052', '17089', '17131', '17183', '17185', '17990', '18222', '18399', '18461', '19034', '19180', '19182', '19183', '19184']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue38644'
versions = ['Python 3.9']
    @vstinner
    Copy link
    Member Author

    Follow-up of bpo-36710 for function calls.

    @vstinner vstinner added 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Oct 30, 2019
    @vstinner
    Copy link
    Member Author

    PR 16997 is a first step. The second step will be to pass tstate to:

    • _PyObject_Vectorcall()
    • _PyObject_MakeTpCall()
    • _Py_CheckFunctionResult()

    (I have a local branch that I have to rewrite on top of PR 16997.)

    @vstinner
    Copy link
    Member Author

    vstinner commented Nov 4, 2019

    New changeset f4b1e3d by Victor Stinner in branch 'master':
    bpo-38644: Add Py_EnterRecursiveCall() to the limited API (GH-17046)
    f4b1e3d

    @vstinner
    Copy link
    Member Author

    vstinner commented Nov 4, 2019

    New changeset be434dc by Victor Stinner in branch 'master':
    bpo-38644: Pass tstate to Py_EnterRecursiveCall() (GH-16997)
    be434dc

    @vstinner
    Copy link
    Member Author

    vstinner commented Nov 5, 2019

    New changeset 1726909 by Victor Stinner in branch 'master':
    bpo-38644: Pass tstate to _Py_CheckFunctionResult() (GH-17050)
    1726909

    @vstinner
    Copy link
    Member Author

    vstinner commented Nov 8, 2019

    New changeset 7e43373 by Victor Stinner in branch 'master':
    bpo-38644: Add _PyObject_VectorcallTstate() (GH-17052)
    7e43373

    @vstinner
    Copy link
    Member Author

    I started a thread on python-dev about this issue:
    "Pass the Python thread state to internal C functions"
    https://mail.python.org/archives/list/python-dev@python.org/thread/PQBGECVGVYFTVDLBYURLCXA3T7IPEHHO/

    @vstinner
    Copy link
    Member Author

    New changeset b9e6812 by Victor Stinner in branch 'master':
    bpo-38644: Add _PyEval_EvalFrame() with tstate (GH-17131)
    b9e6812

    @vstinner
    Copy link
    Member Author

    New changeset 4d231bc by Victor Stinner in branch 'master':
    bpo-38644: Add _PyObject_Call() (GH-17089)
    4d231bc

    @vstinner
    Copy link
    Member Author

    New changeset b5e170f by Victor Stinner in branch 'master':
    bpo-38644: Add _PyEval_EvalCode() (GH-17183)
    b5e170f

    @vstinner
    Copy link
    Member Author

    New changeset 51edf8a by Victor Stinner in branch 'master':
    bpo-38644: Cleanup ceval.h (GH-17185)
    51edf8a

    @vstinner
    Copy link
    Member Author

    One further step would be to change the VECTORCALL/FASTCALL calling convention to pass tstate. But I am not sure what is the risk to do that in Python 3.9? Cython uses FASTCALL internally for example.

    @vstinner
    Copy link
    Member Author

    One further step would be to change the VECTORCALL/FASTCALL calling convention to pass tstate. But I am not sure what is the risk to do that in Python 3.9? Cython uses FASTCALL internally for example.

    I started a thread on python-dev:
    https://mail.python.org/archives/list/python-dev@python.org/thread/PIXJAJPWKDGHSQD65VOO2B7FDLU2QLHH/

    I also wrote an article on this issue, "Pass the Python thread state explicitly":
    https://vstinner.github.io/cpython-pass-tstate.html

    @vstinner
    Copy link
    Member Author

    New changeset 2b1df45 by Victor Stinner in branch 'master':
    bpo-38644: Pass tstate to _Py_FinishPendingCalls() (GH-17990)
    2b1df45

    @vstinner
    Copy link
    Member Author

    New changeset 61f4db8 by Victor Stinner in branch 'master':
    bpo-38644: Pass tstate in ceval.c (GH-18222)
    61f4db8

    @vstinner
    Copy link
    Member Author

    vstinner commented Feb 7, 2020

    New changeset 877ea88 by Victor Stinner in branch 'master':
    bpo-38644: Add Py_EnterRecursiveCall() to python3.def (GH-18399)
    877ea88

    @vstinner
    Copy link
    Member Author

    vstinner commented Feb 7, 2020

    One further step would be to change the VECTORCALL/FASTCALL calling convention to pass tstate.

    My first strong motivation is to get None singleton from tstate. It's now being discussed in bpo-39511: "[subinterpreters] Per-interpreter singletons (None, True, False, etc.)".

    @encukou
    Copy link
    Member

    encukou commented Feb 11, 2020

    New changeset f3fda37 by Victor Stinner in branch 'master':
    bpo-38644: Rephrase What's New entry (GH-18461)
    f3fda37

    @vstinner
    Copy link
    Member Author

    New changeset 08faf00 by Victor Stinner in branch 'master':
    bpo-38644: Add _PySys_Audit() which takes tstate (GH-19180)
    08faf00

    @vstinner
    Copy link
    Member Author

    New changeset 7281898 by Victor Stinner in branch 'master':
    bpo-38644: Pass tstate explicitly in signalmodule.c (GH-19184)
    7281898

    @vstinner
    Copy link
    Member Author

    New changeset 71a3522 by Victor Stinner in branch 'master':
    bpo-38644: Make tstate more explicit inside pystate.c (GH-19182)
    71a3522

    @vstinner
    Copy link
    Member Author

    New changeset 1c1e68c by Victor Stinner in branch 'master':
    bpo-38644: Use _PySys_Audit(): pass tstate explicitly (GH-19183)
    1c1e68c

    @rhettinger
    Copy link
    Contributor

    This adds cost to our most critical code paths. For example, type_call() will now be slower for every single object instantiation.

    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 7, 2021

    Raymond:

    This adds cost to our most critical code paths. For example, type_call() will now be slower for every single object instantiation.

    This issue is now closed. Please open a new issue if you consider that the code should be modified. It would also help to have a benchmark if it's a performance regression ;-)

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @vstinner @encukou @rhettinger