classification
Title: crash on unbounded recursion in __del__ .
Type: Stage:
Components: Interpreter Core Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: tim.peters Nosy List: elaias, gvanrossum, spiv, tim.peters
Priority: low Keywords:

Created on 2001-11-19 15:49 by elaias, last changed 2001-11-27 23:36 by tim.peters. This issue is now closed.

Messages (9)
msg7615 - (view) Author: te (elaias) Date: 2001-11-19 15:49
do the following.

>>> class C:
...     def __del__(self):
...         c = C()
>>> c = C()
>>> d = range(100) #anything really
>>> c = d
>>> c

Segmentation fault (core dumped)

msg7616 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2001-11-19 17:18
Logged In: YES 
user_id=31435

Well, it's not just "an assignment":  the __del__ creates a 
local instance of C, which is destroyed while __del__ is 
trying to return, which calls __del__ again, which creates 
a local instance of C, which is destroyed while __del__ is 
trying to return, which calls __del__ again, etc etc etc.  
So it's roughly the same as

class C:
    def __str__(self):
        return str(self)

print C()

You get unbounded recursion and eventually the stack blows 
up.  "Don't do that" is the best advice <wink>.  Reduced 
priority accordingly; *maybe* Python can be changed to 
detect the stack overflow and give a "maximum recursion 
depth" exception instead.
msg7617 - (view) Author: te (elaias) Date: 2001-11-19 17:45
Logged In: YES 
user_id=109241

Sorry, I thought that this was obvious enough for the person
responsible that I didn't have to explain. Thanks for the
correction though. And I would say that a segfault in any
program is a problem, especially an interpreter.
msg7618 - (view) Author: Andrew Bennetts (spiv) Date: 2001-11-20 14:53
Logged In: YES 
user_id=50945

I agree that segfaults like this are a problem; this is a
potential security problem because it means that untrusted
code can crash the interpreter, even if you try to use
something like rexec.
msg7619 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2001-11-24 02:17
Logged In: YES 
user_id=6380

Let's be clear about rexec.  It is currently a pathetic
excuse for security.  It *could* be made safe, and this is
one of the things that would have to be fixed; but there are
so many other places that I don't really care any more
whether this particular core dump gets fixed (at least not
from a security p.o.v.).

It's strange though that even though there's a Python
function at each level, the Python stack limit check doesn't
trigger before C stack overflows. Maybe it would make more
sense to spend some time getting the general stack limit
check more robust. Unfortunately this has to be done
separately for each C compiler and each operating system
variation... :-(
msg7620 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2001-11-27 21:00
Logged In: YES 
user_id=31435

Assigned to me (to figure out why the Python-level 
recursion check isn't triggering, not to tumble into the 
bottomless "outguess all C runtimes" pit).
msg7621 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2001-11-27 22:37
Logged In: YES 
user_id=31435

OK, Python's recursion check does trigger if "del c" is 
included at the end of the __del__ method.  However, as-is, 
the destruction of c is a side effect of __del__'s frame 
getting decref'ed, and the latter happens at the end of 
PyEval_EvalCodeEx().  eval_frame() is long gone by then, 
and tstate->recursion_depth is checked in the latter.

To be clear, there is a deep chain of Python frames, but 
they're all in the process of being destructed, and tstate-
>recursion_depth has already been decremented (it's just 3 
at the time this blew up on my Windows box):  eval_frame() 
is not on the C stack (except for a few times at the base 
of the C stack).

So, in this sense, it's a failure of the recursion-depth 
hacks to model the true depth of the C stack.  Note there 
are actually 10 levels of C stack for each Python level 
here:  PyEval_EvalCodeEx decrefs the frame, which invokes 
_Py_Dealloc, then to frame_dealloc, which decrefs c and 
triggers _Py_Dealloc -> instance_dealloc, which finds a 
__del__ method and so does PyEval_CallObjectWithKeywords -> 
PyObject_Call -> instancemethod_call -> PyObject_Call -> 
function_call -> PyEval_EvalCodeEx, and we start all over 
again.

No bright ideas here, so unassigned again.
msg7622 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2001-11-27 23:03
Logged In: YES 
user_id=31435

Back to me.
msg7623 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2001-11-27 23:36
Logged In: YES 
user_id=31435

I repaired the tstate->recursion_depth counter for this 
case, in

Python/ceval.c; new revision: 2.291

It doesn't help much, though -- there are so many C stack 
frames per Python level here that sys.setrecursionlimit() 
has to be cranked way down from the default to do any 
good.  Even if you do that, the resulting "maximum 
recursion depth exceeded" exception occurs in a destructor 
so is ignored (you get a msg on a stderr, but the exception 
doesn't propagate outside the __del__ method -- note that 
this is true of *any* exception raised when in a __del__).

Marking this Fixed since it's the best we can do with what 
we've got, and concerned users can force 
sys.setrecursionlimit() to a low value.  I'm not opposed to 
schemes to guess stack limits in some other way, but 
discussion of that doesn't belong in this bug report, and 
we've already got the platform-dependent PyOS_CheckStack() 
gimmick that doesn't appear to be worth having anyway.
History
Date User Action Args
2001-11-19 15:49:44elaiascreate