Title: PyOS_CheckStack does not work
Type: Stage:
Components: Windows Versions: Python 2.6
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: amaury.forgeotdarc Nosy List: amaury.forgeotdarc, kristjan.jonsson, loewis, vstinner
Priority: normal Keywords: patch

Created on 2008-09-29 11:34 by amaury.forgeotdarc, last changed 2008-11-22 20:07 by amaury.forgeotdarc. This issue is now closed.

Messages (6)
msg74024 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-09-29 11:34
On Windows, PyOS_CheckStack is supposed to protect the interpreter from
stack overflow. But doing this, it always crashes when the stack is
nearly full.

The reason is a bad check of the return value of _resetstkoflw():
according to MSDN, the return value is "Nonzero if the function
succeeds, zero if it fails.":

The patch below is enough to replace the "Fatal Python error: Could not
reset the stack!" into a "MemoryError: stack overflow" exception.

Tested with:
>>> loop = None,
>>> for x in xrange(100000): loop = {'x': loop}
>>> len(repr(loop))

Index: Python/pythonrun.c
--- Python/pythonrun.c  (revision 66486)
+++ Python/pythonrun.c  (working copy)
@@ -1749,7 +1755,7 @@
                        EXCEPTION_EXECUTE_HANDLER :
                        EXCEPTION_CONTINUE_SEARCH) {
                int errcode = _resetstkoflw();
-               if (errcode)
+               if (errcode == 0)
                        Py_FatalError("Could not reset the stack!");
msg74044 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2008-09-29 20:27
This issue may be related: issue1069092
msg74052 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-09-29 22:23
Yes, issue1069092 is another case where PyOS_CheckStack is exercised. 
But this other issue was first reported on Unix, when PyOS_CheckStack is 
currently implemented only for Windows.
msg75365 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2008-10-30 09:48
See also issue #3999: I wrote a new generic handler for stack 
overflow "exception".
msg76254 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-11-22 17:09
The patch is fine, please apply it to all versions from 2.6 to 3.0.
msg76257 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-11-22 20:07
Committed r67343 (trunk) and r67344 (release26-maint)
Date User Action Args
2008-11-22 20:07:20amaury.forgeotdarcsetstatus: open -> closed
resolution: accepted -> fixed
messages: + msg76257
2008-11-22 17:09:42loewissetassignee: loewis -> amaury.forgeotdarc
resolution: accepted
messages: + msg76254
nosy: + kristjan.jonsson
2008-10-30 09:48:36vstinnersetmessages: + msg75365
2008-09-29 22:23:00amaury.forgeotdarcsetmessages: + msg74052
2008-09-29 20:27:31vstinnersetnosy: + vstinner
messages: + msg74044
2008-09-29 11:34:10amaury.forgeotdarccreate