Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement CALL_FUNCTION adaptive interpreter optimizations #88691

Closed
Fidget-Spinner opened this issue Jun 28, 2021 · 11 comments
Closed

Implement CALL_FUNCTION adaptive interpreter optimizations #88691

Fidget-Spinner opened this issue Jun 28, 2021 · 11 comments
Labels
3.11 only security fixes pending The issue will be closed if no feedback is provided performance Performance or resource usage

Comments

@Fidget-Spinner
Copy link
Member

Fidget-Spinner commented Jun 28, 2021 •
edited by bedevere-bot

BPO 44525
Nosy @markshannon, @pablogsal, @sobolevn, @Fidget-Spinner, @thesamesam
PRs
  • bpo-44525: Specialize CALL_FUNCTION for C function calls #26934
  • bpo-44525: Specialize simple Python calls. #29033
  • bpo-44525: Add recursive checks for CALL_FUNCTION_BUILTIN_O #29271
  • bpo-44525: Copy freevars in bytecode to allow calls to inner functions to be specialized #29595
  • bpo-44525: Specialize for calls to type and other builtin classes with 1 argument. #29942
  • bpo-44525: Split calls into PRECALL and CALL #30011
  • bpo-44525: Document new call opcodes for 3.11 #30107
  • bpo-44525: Specialize calls to normal python classes #30415
  • bpo-46709: fix race conditions in unittest/test_brake #31273
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2021-06-28.14:22:18.192>
labels = ['3.11', 'performance']
title = 'Implement CALL_FUNCTION adaptive interpreter optimizations'
updated_at = <Date 2022-02-11.11:31:07.338>
user = 'https://github.com/Fidget-Spinner'

    bugs.python.org fields:

    activity = <Date 2022-02-11.11:31:07.338>
actor = 'sobolevn'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2021-06-28.14:22:18.192>
creator = 'kj'
dependencies = []
files = []
hgrepos = []
issue_num = 44525
keywords = ['patch']
message_count = 9.0
messages = ['396639', '404376', '404470', '405108', '405192', '406831', '408554', '408609', '409779']
nosy_count = 5.0
nosy_names = ['Mark.Shannon', 'pablogsal', 'sobolevn', 'kj', 'thesamesam']
pr_nums = ['26934', '29033', '29271', '29595', '29942', '30011', '30107', '30415', '31273']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue44525'
versions = ['Python 3.11']

    Linked PRs
    @Fidget-Spinner
    Copy link
    Member Author

    CALL_FUNCTION can be specialized. Copying from Mark's comments faster-cpython/ideas#54 (comment)

    There are a number of specializations of CALL_FUNCTION that make sense:

1. Calls to a Python function where the arguments and parameters match exactly.
2. Calls to a Python function with keyword arguments, or defaults, when the argument shuffle can be pre-computed.
3. Calls to builtin functions.
4. Calls to specific builtin functions, specifically len and instance.
5. Calls to type with a single argument.
6. Instantiation of a "normal" class, that is a class that does not redefine __new__ and whose metaclass is type.

    I've prepared a PR that should speed up non-keyword calls to PyCFunction. It covers specializations 3 and 4. Stable pyperf microbenchmarks show 5-15% less call overhead for some PyCFunctions. Please see faster-cpython/ideas#54 (comment) for the benchmark script and results.

    This issue is also tied to bpo-44207 (Add a version number to Python functions) which will be required for specializations 1 and 2.

    @Fidget-Spinner Fidget-Spinner added 3.11 only security fixes performance Performance or resource usage labels Jun 28, 2021
    @markshannon
    Copy link
    Member

    New changeset 3163e68 by Ken Jin in branch 'main':
    bpo-44525: Specialize CALL_FUNCTION for C function calls (GH-26934)
    3163e68

    @markshannon
    Copy link
    Member

    New changeset 8863a0f by Mark Shannon in branch 'main':
    bpo-44525: Specialize simple Python calls. (GH-29033)
    8863a0f

    @pablogsal
    Copy link
    Member

    Unfortunately, PR 26934 has broken thes 390x RHEL7 LTO 3.x buildbot as you can see before. As per the buildbot maintenance procedures, we will need to revert this PR unless is fixed in 24 hours.

    @markshannon @Fidget-Spinner

    @markshannon
    Copy link
    Member

    New changeset 0a1a36b by Ken Jin in branch 'main':
    bpo-44525: Add recursive checks for CALL_FUNCTION_BUILTIN_O (GH-29271)
    0a1a36b

    @markshannon
    Copy link
    Member

    New changeset 135cabd by Mark Shannon in branch 'main':
    bpo-44525: Copy free variables in bytecode to allow calls to inner functions to be specialized (GH-29595)
    135cabd

    @markshannon
    Copy link
    Member

    New changeset 9f8f451 by Mark Shannon in branch 'main':
    bpo-44525: Split calls into PRECALL and CALL (GH-30011)
    9f8f451

    @markshannon
    Copy link
    Member

    New changeset 3a60bfe by Mark Shannon in branch 'main':
    bpo-44525: Specialize for calls to type and other builtin classes with 1 argument. (GH-29942)
    3a60bfe

    @markshannon
    Copy link
    Member

    See faster-cpython/ideas#210

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @iritkatriel
    Copy link
    Member

    Is there anything left to do on this?

    @iritkatriel iritkatriel added the pending The issue will be closed if no feedback is provided label Sep 9, 2022
    @pablogsal
    Copy link
    Member

    Closing. Please feel free to re-open if we missed anything.

    @bedevere-bot bedevere-bot mentioned this issue Apr 3, 2023
    Merged
    brandtbucher added a commit that referenced this issue Apr 5, 2023
    @brandtbucher
    GH-88691: Shrink the CALL caches (GH-103230)
    b4978ff
    gaogaotiantian pushed a commit to gaogaotiantian/cpython that referenced this issue Apr 8, 2023
    @brandtbucher @gaogaotiantian
    pythonGH-88691: Shrink the CALL caches (pythonGH-103230)
    0b43d39
    warsaw pushed a commit to warsaw/cpython that referenced this issue Apr 11, 2023
    @brandtbucher @warsaw
    pythonGH-88691: Shrink the CALL caches (pythonGH-103230)
    a568403
    @lemenkov lemenkov mentioned this issue May 30, 2023
    Closed
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.11 only security fixes pending The issue will be closed if no feedback is provided performance Performance or resource usage
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @iritkatriel @markshannon @pablogsal @Fidget-Spinner