classification
Title: Add _testcapi.stack_pointer() to measure the C stack consumption
Type: Stage: resolved
Components: Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2017-07-06 10:51 by vstinner, last changed 2017-09-15 16:38 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
stack_overflow-3.py vstinner, 2017-07-06 10:51
Pull Requests
URL Status Linked Edit
PR 2600 closed vstinner, 2017-07-06 10:51
PR 2601 merged vstinner, 2017-07-06 10:53
PR 2602 closed vstinner, 2017-07-06 10:54
PR 2603 closed vstinner, 2017-07-06 10:56
Messages (6)
msg297822 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-06 10:51
In issues bpo-28870, bpo-29227, bpo-29233 and bpo-29234, I worked on reducing the C stack consumption. I now would to add the _testcapi.stack_pointer() that I used to be able to easily measure the stack consumption.

Try for example attached stack_overflow-3.py to measure the maximum number of C calls before a C stack overflow. The script also computes the average stack usage per C call for a specific kind of C call.
msg297824 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-07-06 11:25
Usually we first merge a patch in master and after that cherry-pick it to other branches.
msg297826 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-06 13:27
> Usually we first merge a patch in master and after that cherry-pick it to other branches.

Right. But in this case, I needed the patch for each branch, to run the test on all branches to write a blog post :-) Here are results (blog post spoiler!):

Table showing the C stack consumption in bytes, and the difference compared to
Python 3.5 (last release before I started working on FASTCALL):

====================  ================  =====  ================  ================
Function                      2.7         3.5          3.6           3.7
====================  ================  =====  ================  ================
test_python_call      1,360 (**+352**)  1,008  1,120 (**+112**)    960 (**-48**)
test_python_getitem   1,408 (**+288**)  1,120  1,168 (**+48**)     880 (**-240**)
test_python_iterator  1,424 (**+192**)  1,232  1,200 (**-32**)   1,024 (**-208**)
Total                 4,192 (**+832**)  3,360  3,488 (**+128**)  2,864 (**-496**)
====================  ================  =====  ================  ================

Table showing the number of function calls before a stack overflow,
and the difference compared to Python 3.5:

====================  ===================  ======  ===================  ===================
Function                       2.7            3.5           3.6           3.7
====================  ===================  ======  ===================  ===================
test_python_call       6,161 (**-2,153**)   8,314   7,482 (**-832**)     8,729 (**+415**)
test_python_getitem    5,951 (**-1,531**)   7,482   7,174 (**-308**)     9,522 (**+2,040**)
test_python_iterator   5,885 (**-916**)     6,801   6,983 (**+182**)     8,184 (**+1,383**)
Total                  17,997 (**-4600**)  22,597  21,639 (**-958**)    26,435 (**+3,838**)
====================  ===================  ======  ===================  ===================
msg297829 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-06 13:52
I just published my blog post: "My contributions to CPython during 2017 Q1"
https://haypo.github.io/contrib-cpython-2017q1.html

See the "Stack consumption" section.
msg298060 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-10 12:37
New changeset 64fa449f559b0e7b0480a177000419a78ebcf3ff by Victor Stinner in branch 'master':
bpo-30866: Add _testcapi.stack_pointer() (#2601)
https://github.com/python/cpython/commit/64fa449f559b0e7b0480a177000419a78ebcf3ff
msg302271 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-15 16:38
I added the function to the master branch.

I changed my mind. I now consider that the function is no needed in other branches. It can be added manually if someone needs it.
History
Date User Action Args
2017-09-15 16:38:44vstinnersetstatus: open -> closed
versions: - Python 2.7, Python 3.5, Python 3.6
messages: + msg302271

resolution: fixed
stage: resolved
2017-07-10 12:37:51vstinnersetmessages: + msg298060
2017-07-06 13:52:43vstinnersetmessages: + msg297829
2017-07-06 13:27:09vstinnersetmessages: + msg297826
2017-07-06 11:25:16serhiy.storchakasetmessages: + msg297824
2017-07-06 10:56:46vstinnersetpull_requests: + pull_request2671
2017-07-06 10:54:28vstinnersetpull_requests: + pull_request2670
2017-07-06 10:53:21vstinnersetpull_requests: + pull_request2669
2017-07-06 10:51:43vstinnersetpull_requests: + pull_request2668
2017-07-06 10:51:19vstinnercreate