Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Control stack usage in large expressions #88012

Closed
markshannon opened this issue Apr 14, 2021 · 4 comments
Closed

Control stack usage in large expressions #88012

markshannon opened this issue Apr 14, 2021 · 4 comments
Assignees
@markshannon
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage

Comments

@markshannon
Copy link
Member

BPO 43846
Nosy @markshannon, @serhiy-storchaka, @pablogsal
PRs
  • bpo-43846: Use less stack for large literals and calls #25403
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/markshannon'
closed_at = <Date 2021-04-21.09:23:59.159>
created_at = <Date 2021-04-14.12:09:31.468>
labels = ['interpreter-core', 'performance']
title = 'Control stack usage in large expressions'
updated_at = <Date 2021-04-21.09:23:59.158>
user = 'https://github.com/markshannon'

    bugs.python.org fields:

    activity = <Date 2021-04-21.09:23:59.158>
actor = 'Mark.Shannon'
assignee = 'Mark.Shannon'
closed = True
closed_date = <Date 2021-04-21.09:23:59.159>
closer = 'Mark.Shannon'
components = ['Interpreter Core']
creation = <Date 2021-04-14.12:09:31.468>
creator = 'Mark.Shannon'
dependencies = []
files = []
hgrepos = []
issue_num = 43846
keywords = ['patch']
message_count = 4.0
messages = ['391072', '391136', '391150', '391502']
nosy_count = 3.0
nosy_names = ['Mark.Shannon', 'serhiy.storchaka', 'pablogsal']
pr_nums = ['25403']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue43846'
versions = []
    @markshannon
    Copy link
    Member Author

    Large literals or function calls with many arguments can consume a lot of stack space.

    This will be a problem for any future work to use a contiguous stack for data and possibly eliminate frame objects for most calls.

    It is also possible (I haven't measured this) that this large stack consumption is hurting performance now, as it might leak memory by leaving giant frames in the free-list or as a zombie frame.

    This fix relatively straightforward. For large literals and argument lists, build them incrementally rather than all at once.

    @markshannon markshannon self-assigned this Apr 14, 2021
    @markshannon markshannon added interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage labels Apr 14, 2021
    @markshannon markshannon self-assigned this Apr 14, 2021
    @markshannon markshannon added interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage labels Apr 14, 2021
    @markshannon
    Copy link
    Member Author

    New changeset 11e0b29 by Mark Shannon in branch 'master':
    bpo-43846: Use less stack for large literals and calls (GH-25403)
    11e0b29

    @serhiy-storchaka
    Copy link
    Member

    What is performance impact of this change? I expect that creating a list incrementally can hurt performance, but how much?

    @markshannon
    Copy link
    Member Author

    I've not measured performance, as the sort of large literals it would impact are very rare, and the sort of calls it would change are almost non-existent.

    I'd be surprised if any performance difference could be reliably detected with our current performance tools.

    This change is an enabler of future optimizations, such as using a contiguous data stack. The impact of those optimizations will swamp any impact of this change, and should be easily measurable.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @markshannon markshannon

    Labels
    interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @serhiy-storchaka @markshannon