classification
Title: Control stack usage in large expressions
Type: performance Stage: resolved
Components: Interpreter Core Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Mark.Shannon Nosy List: Mark.Shannon, pablogsal, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2021-04-14 12:09 by Mark.Shannon, last changed 2021-04-21 09:23 by Mark.Shannon. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 25403 merged Mark.Shannon, 2021-04-14 13:23
Messages (4)
msg391072 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-04-14 12:09
Large literals or function calls with many arguments can consume a lot of stack space.

This will be a problem for any future work to use a contiguous stack for data and possibly eliminate frame objects for most calls.

It is also possible (I haven't measured this) that this large stack consumption is hurting performance now, as it might leak memory by leaving giant frames in the free-list or as a zombie frame.

This fix relatively straightforward. For large literals and argument lists, build them incrementally rather than all at once.
msg391136 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-04-15 13:29
New changeset 11e0b295dee235dd6fd66a10d4823b0dcb014dc4 by Mark Shannon in branch 'master':
bpo-43846: Use less stack for large literals and calls (GH-25403)
https://github.com/python/cpython/commit/11e0b295dee235dd6fd66a10d4823b0dcb014dc4
msg391150 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-04-15 19:32
What is performance impact of this change? I expect that creating a list incrementally can hurt performance, but how much?
msg391502 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-04-21 09:23
I've not measured performance, as the sort of large literals it would impact are very rare, and the sort of calls it would change are almost non-existent.

I'd be surprised if any performance difference could be reliably detected with our current performance tools.

This change is an enabler of future optimizations, such as using a contiguous data stack. The impact of those optimizations will swamp any impact of this change, and should be easily measurable.
History
Date User Action Args
2021-04-21 09:23:59Mark.Shannonsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-04-21 09:23:47Mark.Shannonsetmessages: + msg391502
2021-04-15 19:32:54serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg391150
2021-04-15 13:29:20Mark.Shannonsetmessages: + msg391136
2021-04-14 13:23:44Mark.Shannonsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request24136
2021-04-14 13:23:32Mark.Shannonsetnosy: + pablogsal
2021-04-14 12:09:31Mark.Shannoncreate