I suspect that there is some deeper (more severe) issue behind the
problem I describe below. I've observed the following on Windows only.
Didn't try it with different OSs

running dragbug.py shows different behaviour with Python 3.1 compared to
Python 2.6:

Running it with Python 3.1 and performing heavy turtle dragging with the
mouse results in:

Fatal Python error: Cannot recover from stack overflow
This application has requested the Runtime to terminate it in an unusual
way. Please contact the application's support team
-- As I do at the moment ;-) 

Running the same script from Python 2.6: The error is much harder to
reproduce (only with very excessive dragging). It's a bit easier to
reproduce when running the program from a console and  if it occurs one
gets a conventional Python error message:

First a long sequence of:
Exception in Tkinter callback
Traceback (most recent call last):
Exception RuntimeError: 'maximum recursion depth exceeded while .... 
ignored

Followed by:

Traceback (most recent call last):
  File "dragbug.py", line 10, in <module>
    mainloop()
  ....
  File "c:\Python26\lib\traceback.py", line 57, in print_tb
    if hasattr(sys, 'tracebacklimit'):
AttributeError: 'module' object has no attribute 'tracebacklimit'

This problem can be overcome by setting a higher recurson limit. But I
think one should require that it doesn't produce a Fatal Error like in
Python 3.1

Regards,
Gregor

With py3k trunk on Gentoo Linux after less than a second of flailing the
cursor around on the screen I got:

rdmurray@partner:~/python/py3k>./python dragbug.py 
Fatal Python error: Cannot recover from stack overflow.
zsh: abort      ./python dragbug.py

For anyone trying this on another system: you must specifically click on
and drag the mouse/turtle graphic.

Running from IDLE edit, I get no traceback in the Shell Window, unlike
with the script I attached to #7074 and now here. While I am closing the
latter as probably due to the same cause, it does act differently:
adding the 'speed(0)' command present in dragbug.py appears to have
fixed the problem with turtlecrash.py.

Rewriting dragbug to use tkinter canvas directly would determine whether
the turtle module is a factor. The change of behavior with turtlecrash
with the speed command suggests to me that it might be.

I've written and appended with this message a small script using
tkinter, that produces the same crash as described earlier with turtle.py:

tkinter_recursion_31.py

It occurs when heavily and fast dragging the red square. So the bug
doesn't depent on turtle.py

I'd like to mention again, that the corresponding script under Python26
does crash also, but with a regular Python Error-Message. It does not
kill IDLE when stated from within IDLE.

So I'd suggest to give this one a higher priority. Should be fixed with 3.2

Regards,
Gregor

I can confirm the crash persists as of Python 3.1.3 on Windows, and would like to add my vote to prioritizing it.  

Without having delved into the code, it seems strange that the rapid stream of events is causing stack overflow / recursion limit problems.

It suggests that tkinter is allowing new event arrivals to interrupt older event handling, so that the older ones remain incomplete as the newer ones pile up on the stack. If this is the case, it is going to lead to long-term instability, and needs attention.

The usual technique is to use a queue to decouple "occurs now" from "handle immediately". (The OS puts the mouse and keyboard events onto window's event queue).  So the mainloop that services the event queue should not start handling a new queued event until the previous handling has completed.

Attached is a crashing program that shows that the event handler is called again before activation of the prior instance has completed.

I also have a second turtle that queues the nested events.  It doesn't crash the system, at least.

Perhaps someone can also explain why the handler doesn't work if I leave out the global declaration.

Oops, I wish I hadn't asked that silly question about the global declaration!

Here is the tweaked file...

Issue reproduceable on 3.2 (r32:88445) with drag_bug_is_nesting_events.py

I ran into a similar issue and believe I have a simpler repro.  I think the issue here might be that when you take a stack overflow inside of a sys.settrace handler that you can then later violate the stack depth.  The attached test.py has no dependencies and crashes 3.x but works on 2.x.

Yes, this is all by design. The interpreter *has* to stop: either it stops in a controlled way (the fatal error) or the stack is blown and it crashes. If you think the fatal error (basically a C abort() call) should be replaced with another way of exiting, please suggest so.

(actually, it's not the interpreter as a whole, only the current thread, but crashing a thread without affecting the others is probably impossible, due to reference leaks, resource cleanup, etc.)

Antoine: If you're looking at my test.py then my expectation is that this doesn't crash because a RuntimeError should be raised when the maximum recursion limit is hit, and then the trace handler should be uninstalled because it leaks an exception.  And that's exactly what seems to happens on Python 2.x.  We shouldn't ever hit the OS stack limit because Python's recursion limit should be enforced even in the face of a sys.settrace handler.

> Antoine: If you're looking at my test.py then my expectation is that
> this doesn't crash because a RuntimeError should be raised when the
> maximum recursion limit is hit, and then the trace handler should be
> uninstalled because it leaks an exception.

I don't understand why you say that "the trace handler leaks an
exception", since it silences it in the "try ... except" block.

It's catching the exception when it invokes x, but the recursion enforcement should happen at a method prolog, including at the invocation of g.  Therefore if we're at or beyond the recursion limit when invoking the trace handler the limits should still be enforced and that should be the same as the trace handler raising.

> It's catching the exception when it invokes x, but the recursion
> enforcement should happen at a method prolog, including at the
> invocation of g.  Therefore if we're at or beyond the recursion limit
> when invoking the trace handler the limits should still be enforced
> and that should be the same as the trace handler raising.

That's where 3.x is different: 3.x temporarily bumps up the recursion
limit a bit when it is first reached, in order to let various cleanup
handlers run as intended. This is a nice thing in the general case, but
means it can degenerate in more involved or desperate cases.

(although here it's not clear to me why a second recursion error occurs
after the first one)

Maybe there just needs to be a max that it will bump it up?  

FYI this isn't actually causing any problems for me, I just ran into it while doing IronPython development and was surprised to be able to crash the interpreter w/ pure Python code, and my crash looked awfully similar to this bug.

> FYI this isn't actually causing any problems for me, I just ran into it 
> while doing IronPython development and was surprised to be able to
> crash the interpreter w/ pure Python code, and my crash looked awfully 
> similar to this bug.

I agree that crashing isn't ideal, and there may be more graceful ways of crashing instead of abort(), so that people don't get that horrible message box under Windows. Suggestions welcome.

One thought might be to do a recursion check (and maybe for multiple frames) when entering a try rather than incrementing the recursion limit to allow the handlers to run.  

That would cause the exception to be more likely taken before you run the code which needs some form of cleanup and then maybe the recursion limit could be a hard limit which can't be increased forever.

New changeset 0dfa3b09a6fe by Nick Coghlan in branch '3.2':
Record a known crasher from #6717
http://hg.python.org/cpython/rev/0dfa3b09a6fe

New changeset 509f7a53f8cc by Nick Coghlan in branch '3.3':
Merge #6717 crasher from 3.2
http://hg.python.org/cpython/rev/509f7a53f8cc

New changeset 47943fe516ec by Nick Coghlan in branch 'default':
Merge #6717 crasher from 3.3
http://hg.python.org/cpython/rev/47943fe516ec