Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

co_stacksize estimate can be highly off #68528

Closed
arigo mannequin opened this issue May 31, 2015 · 8 comments
Closed

co_stacksize estimate can be highly off #68528

arigo mannequin opened this issue May 31, 2015 · 8 comments
Labels
3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage

Comments

@arigo
Copy link
Mannequin

arigo mannequin commented May 31, 2015

BPO 24340
Nosy @nascheme, @pitrou, @benjaminp, @methane, @serhiy-storchaka, @1st1
PRs
  • bpo-24340: Fixed a "leak" in the computation of the stack size effect #1938
  • bpo-24340: Fix estimation of the code stack size. #5076
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-01-11.19:28:17.793>
created_at = <Date 2015-05-31.18:29:54.905>
labels = ['interpreter-core', '3.7', 'performance']
title = 'co_stacksize estimate can be highly off'
updated_at = <Date 2018-01-11.19:28:17.792>
user = 'https://github.com/arigo'

    bugs.python.org fields:

    activity = <Date 2018-01-11.19:28:17.792>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2018-01-11.19:28:17.793>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2015-05-31.18:29:54.905>
creator = 'arigo'
dependencies = []
files = []
hgrepos = []
issue_num = 24340
keywords = ['patch']
message_count = 8.0
messages = ['244552', '295118', '295119', '295122', '305677', '309346', '309347', '309733']
nosy_count = 7.0
nosy_names = ['nascheme', 'pitrou', 'benjamin.peterson', 'Arfrever', 'methane', 'serhiy.storchaka', 'yselivanov']
pr_nums = ['1938', '5076']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue24340'
versions = ['Python 3.7']
    @arigo
    Copy link
    Mannequin Author

    arigo mannequin commented May 31, 2015

    The computation of co_stacksize' by the compiler is known to give only an upper bound estimate. http://bugs.python.org/issue1754094 is an example of fixing a "leak" where every repetition of a statement makes co_stacksize' bigger by 1. However, in the whole 3.x series (from at least 3.2), the following simple code, which at runtime requires about 4 or 5 stack slots, leaks 14(!) slots for every `try:' block. Maybe this should be improved to reduce the extreme size of the leak.

    def g():
    try: pass
    except ImportError as e: pass
    try: pass
    except ImportError as e: pass
    try: pass
    except ImportError as e: pass
    ...

    i.e. any function that is big enough to contain 6 try: blocks in sequence will have its stack size overestimated by about 70.

    @arigo arigo mannequin added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label May 31, 2015
    @pitrou pitrou added the performance Performance or resource usage label Jun 1, 2015
    @arigo arigo mannequin added the 3.7 (EOL) end of life label Jun 4, 2017
    @serhiy-storchaka
    Copy link
    Member

    PR 1938 tries to balance the stack effect computation for try/except and try/finally. I'm not sure that it is worth backporting.

    Noised Antoine and Benjamin as people related to bpo-3021.

    @pitrou
    Copy link
    Member

    pitrou commented Jun 4, 2017

    I'm against backporting performance improvements which don't fix a severe regression.

    @serhiy-storchaka
    Copy link
    Member

    This isn't so easy. Seems the simplest way to solve this issue is implementing bpo-17611.

    @nascheme
    Copy link
    Member

    nascheme commented Nov 6, 2017

    The WIP pull request PR# 2827 seems to help. The following code prints 86 on python3.6 and 25 with PR 2827 applied.

    def g():
    try: pass
    except ImportError as e: pass
    try: pass
    except ImportError as e: pass
    try: pass
    except ImportError as e: pass
    try: pass
    except ImportError as e: pass
    try: pass
    except ImportError as e: pass
    try: pass
    except ImportError as e: pass
print(g.__code__.co_stacksize)

    @serhiy-storchaka
    Copy link
    Member

    With PR 5076 the result of the above example is 10.

    @serhiy-storchaka
    Copy link
    Member

    Tests originally based on Antoine's tests added for PR 2827.

    @serhiy-storchaka
    Copy link
    Member

    New changeset d4864c6 by Serhiy Storchaka in branch 'master':
    bpo-24340: Fix estimation of the code stack size. (bpo-5076)
    d4864c6

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @nascheme @pitrou @serhiy-storchaka