classification
Title: stack overflow after hitting recursion limit twice
Type: crash Stage:
Components: Interpreter Core Versions: Python 3.0, Python 3.1
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: pitrou Nosy List: georg.brandl, ggenellina, loewis, pitrou
Priority: normal Keywords: patch

Created on 2009-02-28 11:26 by ggenellina, last changed 2009-03-13 21:00 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
issue5392.patch pitrou, 2009-03-01 17:18
Messages (8)
msg82906 - (view) Author: Gabriel Genellina (ggenellina) Date: 2009-02-28 11:26
Set sys.setrecursionlimit to 50 or lower. Then, the second time the 
recursion limit is reached, the interpreter crashes with a stack 
overflow.
This happens both with released 3.0.1 and the py3k branch, on Windows.
At least on my PC, 51 appears to be the minimum acceptable value for 
sys.setrecursionlimit.

Python 3.1a0 (py3k, Feb 28 2009, 04:16:04) [MSC v.1500 32 bit (Intel)] 
on win32
Type "help", "copyright", "credits" or "license" for more information.
p3> import sys
p3> sys.setrecursionlimit(20)
p3> def g(): g()
...
p3> g()
Traceback (most recent call last):
...
RuntimeError: maximum recursion depth exceeded
p3> g()
Fatal Python error: Cannot recover from stack overflow.

This application has requested the Runtime to terminate it in an 
unusual way.
Please contact the application's support team for more information.

C:\APPS\python\py3k\PCbuild>
msg82909 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-02-28 14:37
I can reproduce that (with the same limits) on Linux here.
It doesn't happen with 2.6.

(Although it is technically not a crash but a controlled abort().)
msg82916 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-02-28 15:32
This is probably due to the recursion overflow recovery code in py3k,
which has a hard-wired constant of 50 somewhere :-)

(is setting the recursion limit so low a requirement for your
application? or were you just experimenting with it? as Georg said, it's
not a crash but a deliberate fatal error... although we can probably
change the behaviour when the recursion limit is set to 50 or lower)
msg82967 - (view) Author: Gabriel Genellina (ggenellina) Date: 2009-03-01 15:58
It is an artificial value, I don't require a recursion 
limit so low in any application. I found it when 
looking into #5370.
If there is a lower limit to sys.setrecursionlimit, 
maybe it should be enforced. But since it fails only 
the second time, it looks like a bug somewhere.
msg82968 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-01 16:44
The fact it fails only the second time is by design, although I'm not
sure the design is useful, and it's probably not documented anywhere.
The error message says it : "Cannot *recover* from stack overflow",
which means there was a first stack overflow, and Python thinks it has
failed recovering from it since a second stack overflow occurred
afterwards.

We could probably make the recovery detection smarter.
msg82969 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-01 17:18
Here is a patch. I've put the tests in test_sys.py, I haven't found a
better place for them.
msg82979 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-03-01 19:47
> The fact it fails only the second time is by design, although I'm not
> sure the design is useful, and it's probably not documented anywhere.

It helped me debug a number of interpreter crashes in 3.0. When a stack
overflow occurred, in certain cases, the interpreter would catch the
exception, and consider it as failure in the callback it tried to invoke
(e.g. when invoking __eq__ during dictionary lookups). Rather than
letting the stack unwind, it would continue to let the stack overflow,
and eventually managed to crash the entire process. The recovery check
is there to detect that, after a stack overflow, it really unwound a
sufficient number of stack frames, rather than overflowing again and
again.
msg83533 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-13 19:25
Committed in r70344.
History
Date User Action Args
2009-03-13 21:00:44pitrousetstatus: pending -> closed
2009-03-13 19:25:52pitrousetstatus: open -> pending

messages: + msg83533
resolution: fixed
2009-03-01 19:47:33loewissetmessages: + msg82979
2009-03-01 17:18:21pitrousetfiles: + issue5392.patch
keywords: + patch
messages: + msg82969
2009-03-01 16:59:25pitrousetpriority: normal
assignee: pitrou
2009-03-01 16:44:21pitrousetmessages: + msg82968
2009-03-01 15:58:09ggenellinasetmessages: + msg82967
2009-02-28 15:32:14pitrousetmessages: + msg82916
2009-02-28 14:37:18georg.brandlsetnosy: + georg.brandl
messages: + msg82909
2009-02-28 14:25:59benjamin.petersonsetnosy: + loewis, pitrou
2009-02-28 11:26:43ggenellinacreate