classification
Title: IDLE - shell becomes unresponsive if debugger windows is closed while active.
Type: behavior Stage: resolved
Components: IDLE Versions: Python 3.6, Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: THRlWiTi, asvetlov, jimbo1qaz, markroseman, python-dev, roger.serwy, terry.reedy
Priority: normal Keywords: patch

Created on 2012-07-13 19:38 by roger.serwy, last changed 2019-09-19 17:33 by terry.reedy. This issue is now closed.

Files
File name Uploaded Description Edit
issue15348.patch roger.serwy, 2012-10-08 07:09 review
stable_idle_debugger.diff roger.serwy, 2012-10-09 03:21 review
Messages (18)
msg165411 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-07-13 19:38
The IDLE shell does not respond to commands if the debugger window is closed by clicking "X". This is due to PyShell's "executing" flag not being reset.

Steps to reproduce:

1) Start IDLE with a shell.
2) Enable debugger.
3) Press enter in the shell. (activates the debugger)
4) Close the debugger by clicking "X".

The shell is no longer responsive to new commands. Selecting "Restart Shell" does not fix the problem either.
msg172363 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-10-08 07:09
Attached is a patch against 3.4 to solve the issue.

The debugger was originally written for running IDLE without a subprocess. As a result, calls to idb.run() from Debugger.py would block until completed.  If .interacting == 1 then clicking "X" would not close the debugger window. Once the debugger finishes, the .interacting flag would reset to 0, which then allows for the Debugger.close() method to destroy the debugger window. Clicking "Quit" is one way to have the debugger finish.

When running *with* a subprocess, the idb.run() returns immediately, since it is using the IdbProxy object instead. As a consequence, the .interacting flag is also reset to 0 immediately despite the debugger still being active. Thus, clicking "X" would allow the debugger to close without properly stopping the debugger via clicking "Quit".

A useful side-effect of the interactive debugger being active is that the PyShell.executing flag is set to true. When the debugger stops, the call to PyShell.endexecuting() from poll_subprocess() resets the .executing flag. The patch checks if PyShell's executing is true in order to prevent the Debugger Window from being closed by clicking "X". This makes the behavior is consistent with running the IDLE debugger without a subprocess.


When running IDLE without a subprocess is fully removed (see issue16123), then maintaining the Debugger code will become simpler. Until then, testing against both code paths is necessary.
msg172439 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-10-09 03:21
While trying to address #15347, I discovered one too many corner cases where the debugger breaks IDLE. The stable_idle_debugger.diff against 3.4 contains necessary changes to make IDLE more reliable while debugging. Since the IDLE debugger is not documented these changes should not be controversial.

Here are all the cases the patch fixes:
1) Closing the debugger while debugging causes the shell to not respond to new commands when using a subprocess.
2) Pressing F5 in an editor several times with an active debugger causes IDLE to become unresponsive. 
2.1) When not using the subprocess, the patch fixes an AttributeError in runcode in PyShell.py, since .interp is not an attribute of the interpreter.
3) Closing the shell window while the debugger is debugging prevents IDLE from terminating if no editors are open. #15347
4) Closing the shell with an idle debugger causes a traceback in the shell (with no subprocess).

The patch creates the following behavior:

1) The debugger cannot be closed by clicking "X" while it is debugging. Instead, the "Quit" button gets flashed as a reminder to stop the debugger before closing.
2) Pressing F5 in the editor window while the debugger is debugging brings focus to the debugger window.
3) Closing the shell will close the debugger properly. The .close() method of Debugger.py quits the nested eventloop in .interaction() if needed.

I tested these issues and solutions on Linux. Can someone review the patch for Windows and Mac?
msg172440 - (view) Author: jimbo1qaz (jimbo1qaz) Date: 2012-10-09 03:23
Is it possible to make the X button quit the debugger if enabled?
msg172441 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-10-09 03:26
Clicking "X" while the debugger is enabled, but not actively debugging a program, will close the debugger window.
msg172442 - (view) Author: jimbo1qaz (jimbo1qaz) Date: 2012-10-09 03:28
What's the intended behavior? stop debugging and quit the program? stop debugging and continue freerunning? I'm using 3.3, can you make the patch stop debugging the active running program when you click the X?
msg172443 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-10-09 03:34
If the debugger is active, then clicking "X" will flash the "Quit" button. You must click the quit button first before closing the debugger window.
msg172444 - (view) Author: jimbo1qaz (jimbo1qaz) Date: 2012-10-09 03:35
Then frigging change it!
msg172445 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-10-09 03:38
Closing the active debugger with "X" creates a problem with references and callbacks. I encountered too many corner cases where I could not implement that behavior simply.
msg172446 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-10-09 03:43
But of course, you are more than welcomed to try to submit a patch yourself. Just make sure that those corner cases I described earlier are handled reasonable when running IDLE with and without a subprocess.
msg172447 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-10-09 04:15
I have never used the debugger, but I verified the bug. While something was 'hung', the debugger would not restart because something was 'busy'. I also verified that clicking [quit] first and then [x] works. I will try to test the second patch sometime.

Jimbo, the first priority is to fix crashes and hangs. We control the response to buttons we add, but the [x] close button involves the gui and operating system. So working with it can be trickier than it seems it ought to be.
msg245520 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-06-19 19:53
#24455 is probably related.
msg255002 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-11-20 17:29
"not responsive to new commands" means that a statements can be typed but <Enter> is ignored and no execution takes place.
msg255035 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-11-21 01:38
The patch for #24455 replaced root.mainloop,.quit with tcl vwait, set as the suspend/resume mechanism for Debugger.interation.  After this, closing the shell as described in #15347, or the debugger with [x] as described here, on my Windows 10 machine, resulted in

  File "F:\Python\dev\34\lib\idlelib\Debugger.py", line 20, in user_line
    self.gui.interaction(message, frame)
_tkinter.TclError: invalid command name ".86740760.86830984.86827736"

instead of a hang.  This is an improvement as the Exception can be caught and ignored.  People testing of other systems did not report this.  With TclError caught, closing the debugger with [x] then produced

  File "F:\Python\dev\34\lib\idlelib\Debugger.py", line 21, in user_line
    self.gui.interaction(message, frame)
RuntimeError: Unknown object id: 'gui_adapter'

I applied the patch with Runtime also caught and ignored.

The exceptions did not occur when [Quit] was pressed first.  So I think [x] should do whatever Quit does before closing.
msg255040 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-11-21 05:03
(I was wrong about seeing the exception when closing the shell.)

A major difference between [Quit] and [x] is that [Quit] stops the running program whereas [x] closes the visible gui but leaves the program to run through to the end, as though the [go] button were pressed.  I added 'as e' and 'print(e)' to the try except and there are multiple RuntimeErrors, 1 per line traced by [step], including IDLE code for input() and print() calls, with [x], but none with [quit] first.

The TclError only occurs in 3.4.3, not in 2.7.10.

The string 'gui_adapter' comes from RemoteDebugger.py, line 28.
  gui_adap_oid = "gui_adapter"
It represents the debugger window in the idle process.  My interpretation of the traceback is that when
  self.gui.interaction(message, frame)
is executed in the user process, the interaction call actually calls debugger.interaction in the idle process via RemoteDebugger.GUIProxy.interaction.  The latter does not appear in the traceback because IDLE rpc calls are deleted from tracebacks before they are printed.

Replacing self.abort_loop with self.quit (which calls abort_loop) eliminates the Runtime exceptions, as the backend debugger in the user process is stopped before the idle process frontend gui disappears.

However, self.idb.set_quit (within self.quit) raises 
  AttributeError: 'Idb' object has no attribute 'botframe'
if one attempts to immediately close the debugger window (with menu or button) before it is activated.  It is then left as a zombie window. The exception is in the Tkinter callback, so it is printed to console rather than the IDLE shell, and included RemoteDebugger and rpc calls, but not the line in Bdb.set_quit(self), 
        self.stopframe = self.botframe
that actually raised the exception.  So the call must either be conditioned or the exception caught.  I decided to go with latter now.
msg255041 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-11-21 05:05
New changeset 427ad0ede016 by Terry Jan Reedy in branch '2.7':
Issue 15348: Stop debugger engine (normally in user process)
https://hg.python.org/cpython/rev/427ad0ede016

New changeset 4eae64a9cd26 by Terry Jan Reedy in branch '3.4':
Issue 15348: Stop debugger engine (normally in user process)
https://hg.python.org/cpython/rev/4eae64a9cd26
msg255042 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-11-21 05:26
I don't like the idea of [x] flashing quit instead of closing.  This patch, on top of the one for #24455, may not be perfect, but the two are definite improvements.  Unless I see something critically bad first, I want them in the upcoming releases.  Mark claims that his second patch for #15347 improves behavior for no subprocess.  But that was without this patch.

There is one situation not mentioned that I know is still glitchy.  Debug a program with input(prompt).  Run up to and including that line.  While the program is waiting for input, close with [x].  The shell prints '[DEBUG OFF]' after the prompt and '>>> ' on the next line.  But that is a fake prompt in that a statement entered will not be executed, but will be seen as the response to the pending input().

While there is a pending input (or any other blocked statement), the [Quit] button is disabled.  The close method should detect this 'program executing' condition and ask whether to kill it, just as if one tries to close IDLE under the same condition.

I am leaving this open, at least for now, at least for this.
msg352809 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-09-19 17:33
The input glitch mentioned above no longer exists, so closing.
History
Date User Action Args
2019-09-19 17:33:03terry.reedysetstatus: open -> closed
resolution: fixed
messages: + msg352809

stage: needs patch -> resolved
2015-11-21 05:26:07terry.reedysetmessages: + msg255042
2015-11-21 05:06:00python-devsetnosy: + python-dev
messages: + msg255041
2015-11-21 05:03:37terry.reedysetmessages: + msg255040
versions: + Python 3.6
2015-11-21 01:38:28terry.reedysetassignee: roger.serwy -> terry.reedy
messages: + msg255035
stage: patch review -> needs patch
2015-11-20 17:29:09terry.reedysetmessages: + msg255002
2015-09-03 21:32:46markrosemansetnosy: + markroseman
2015-06-23 23:47:06THRlWiTisetnosy: + THRlWiTi
2015-06-19 19:53:47terry.reedysetmessages: + msg245520
2015-06-02 23:37:52terry.reedylinkissue24367 superseder
2014-10-03 03:36:06terry.reedysetversions: + Python 3.5, - Python 3.3
2013-04-03 05:55:15roger.serwysetassignee: roger.serwy
versions: + Python 3.4, - Python 3.2
2012-10-09 04:15:27terry.reedysetmessages: + msg172447
2012-10-09 03:43:54roger.serwysetmessages: + msg172446
2012-10-09 03:38:57roger.serwysetmessages: + msg172445
2012-10-09 03:35:20jimbo1qazsetmessages: + msg172444
2012-10-09 03:34:59roger.serwysetmessages: + msg172443
2012-10-09 03:28:28jimbo1qazsetmessages: + msg172442
2012-10-09 03:26:42roger.serwysetmessages: + msg172441
2012-10-09 03:23:53jimbo1qazsetmessages: + msg172440
2012-10-09 03:21:28roger.serwysetfiles: + stable_idle_debugger.diff

messages: + msg172439
stage: patch review
2012-10-08 07:09:05roger.serwysetfiles: + issue15348.patch

nosy: + terry.reedy, asvetlov
messages: + msg172363

keywords: + patch
2012-10-08 00:16:28jimbo1qazsetnosy: + jimbo1qaz
2012-10-08 00:14:18roger.serwylinkissue16159 superseder
2012-07-13 19:38:40roger.serwysettype: behavior
2012-07-13 19:38:09roger.serwycreate