Consider the following code, typed interactively:

>>> import time
>>> time.sleep(1e6)

This will sleep for a bit over one and a half weeks.  If this was typed in error, you may want to interrupt it.  If using the command line, this is easy: just use Ctrl-C.  If using IDLE, Ctrl-C has no effect.  One could attempt to restart the shell with Ctrl-F6, which seems to work, but in fact the process remains in the background, hung until the timeout expires.  There are two obvious workarounds: one is to sleep in a separate thread, so as to avoid blocking the main thread, and the other is to use a loop with smaller sleep increments:

for ii in range(1e5): sleep(10)

Now it only takes 10 seconds to interrupt a sleep.  But these are both clumsy workarounds.  They're so clumsy that I think I'm not going to use IDLE for this particular program and just use python -I.  Would be nice if this were fixed.

I verified on Win 10 with 3.6.1.  What system are you running on?

An automatic unittest test would be nice, but I cannot imagine how to write one.  Even a human-verified htest (IDLE private term) would  be good.  Maybe I should write a live-interaction test script.

^C is bound to pyshell.PyShell.cancel_callback.  When code is executing, this calls pyshell.ModifiedInterpreter.interrupt_subprocess.  In a new thread, this starts pyshell.ModifiedInterpreter.__request_interrupt.  I verified the calls thus far, with debug prints, while the user execution thread is sleeping.  __request_interrupt sends vua rpc ('exec', 'interrupt_the_server').

'interrupt_the_server' refers to a method of run.Executor:
       if interruptable:
            _thread.interrupt_main()

 Here I am unsure which thread this executes in and when.  Interrupting "while True: a=1" is no problem.  Does interrup_main not work while the main thread is sleeping, or does the above not get executed?

Scratch my confusion.  I added "print('after')" immediately after _thread.interrupt_main() in idlelib.run.Executor.interrupt_the_server() and 'after' is printed in Shell right after I hit ^C but a time.sleep(10) is not immediately interrupted and 'KeyboardInterrupt' does not show for nearly 10 seconds.  I don't know if this is a bug or unavoidable limitation not mentioned in the _thread doc.  I might be system dependent.
https://docs.python.org/3/library/_thread.html#_thread.interrupt_main

It's simple to fix this in Python 3 on Windows. PyErr_SetInterrupt in Modules/signalmodule.c needs the following addition:

    #ifdef MS_WINDOWS
        SetEvent(sigint_event);
    #endif

In the main thread on Windows, time.sleep() waits on this event. 

On Unix, time.sleep() uses select(). We could interrupt it by signaling the process, or explicitly the main thread, via kill() or pthread_kill(). See issue 21895 for a related discussion.

Eryk, would the addition go before or after "trip_signal(SIGINT);"

I posted "Issue with _thread.interrupt_main (29926)" on pydev list.  Steven D'Aprano verified delayed response on *nix.

Martin Panter said
"Looking at the implementation, _thread.interrupt_main just calls PyErr_SetInterrupt. It doesn’t appear to send a signal. I played with “strace” and couldn’t see any evidence of a signal. I guess it just sets a flag that will be polled. To actually interrupt the “sleep” call, you might need to use “pthread_kill” or similar (at least on Unix)."

For Windows the event should be set after trip_signal(SIGINT). That way the flag is guaranteed to be tripped when PyErr_CheckSignals is called from the time module's pysleep() function. This in turn calls the default SIGINT handler that raises a KeyboardInterrupt.

If you want to trigger the standard signal handling logic, then raise(SIGINT) is also an option. On unix it probably won't help because of issue 21895, but using raise() here + fixing 21895 by using pthread_kill in the c level signal handler would together give a pretty simple and robust way to pretend that the user hit control-C.

But... interrupt_main isn't documented to raise a SIGINT, it's documented to raise KeyboardInterrupt. These are very different, e.g. if the user has installed a custom handler or even set SIGINT to SIG_IGN or masked it. (In the later two cases, using pthread_kill to send a signal to the main thread *won't* interrupt any ongoing syscall.) This is *much* more difficult to make work correctly. And actually what IDLE wants is a fake control-C anyway!

So I'd suggest either redefining the semantics of interrupt_main to be "acts as if SIGINT was received" or else abandoning interrupt_main to its slightly-buggy semantics and adding a new function that does raise(SIGINT) and switching IDLE to use this new function. And fix issue 21895 in either case.

(oh, in case it wasn't obvious: the advantage of raise() over kill() and pthread_kill() is that raise() works everywhere, including Windows, so it would avoid platform specific logic. Or if you don't like raise() for some reason then you can get the same effect by doing handler = PyOS_getsig(SIGINT); if (handler) handler(SIGINT); )

(Also, this is all assuming the python 3 logic on Windows. On Windows versions of python 2 I can't see any way to make time.sleep interruptible without significant rewriting. The problem is that on py2 the special Event used to signal the arrival of a control-C is a private variable that gets hooked up directly to Windows's low-level control-C logic, and there's effectively no way to get at it from outside. You might think GenerateConsoleCtrlEvent would help, but it is broken in too many ways for me to fit in this text box.)

Nathaniel's suggestion to update the implementation to work with raise(SIGINT) sounds like a good idea.

Fpr Windows, GenerateConsoleCtrlEvent is definitely a non-starter. PyErr_SetInterrupt shouldn't depend on having an attached console. IDLE generally runs without one. Even if the process has a console, there's no way to limit GenerateConsoleCtrlEvent to just the current process. It works with process groups, like POSIX killpg(). Typically a process is created in the session's Winlogon process group unless a new group was created by calling CreateProcess with the flag CREATE_NEW_PROCESS_GROUP. If you call GenerateConsoleCtrlEvent on a process ID that's not a group ID, the console behaves as if you passed group ID 0, which broadcasts to all processes attached to the console.

Even is IDLE is started from a console, the user execution process is not attached to one.

What I want, from an IDLE perspective, is for ^C in IDLE's shell to work as well as it does in a terminal.  That means either fixing _thread.interrupt_main to always do what it usually does (from a user perspective), or to be given something else to use in .interrupt_the_server() that does so.

The current workaround for stopping runaway code in interactive mode, restarting the shell with cntl-F6 or the menu entry Shell => Restart Shell, is a) obscure and not known to most beginners, and b) does too much in that it throws away the workspace.  It is much like closing a terminal Window with [X], and restarting the terminal and python, instead of using ^C.

Viktor, I am nosying you because of your posts to #21895.  Do you have any comments on this issue?

> Even is IDLE is started from a console, the user execution process 
> is not attached to one.

Actually, the execution process is attached to a console, but it doesn't use it for sys.std*. Try

    open('con', 'w').write('spam\n')

> It is much like closing a terminal Window with [X], and restarting 
> the terminal and python, instead of using ^C.

There's no default handler for SIGBREAK, so when working in the console it's typically better to use Ctrl+Break to kill Python. This preserves the console's arrow-key & F7 input history and scrollback buffer.

Hi Terry, this patch is what I imagined a fix would look like for Linux. I am not familiar with Idle (internally nor externally), so there may be improvements you can make.

It works as I expected for normal blocking functions and a tight “for” loop: it interrupts any blocking call, and triggers the usual SIGINT handler. If SIGINT has a Python handler (e.g. by default), that gets called which usually raises KeyboardInterrupt.

My change has a major flaw: it seems to deadlock something if you interrupt “input()” or “sys.stdin.readline()”. Perhaps you might have a better clue what the problem is. With the default SIGINT handler, this is what I see in the Idle window:

>>> input()  # Press Ctrl+C
Traceback (most recent call last):  <-- cursor flashing at end of line

If SIGINT is ignored, or the Python handler doesn’t raise an exception, Ctrl+C seems to have the effect of inputting a newline:

>>> input()  # Press Ctrl+C
''
>>> sys.stdin.readline()  # Press Ctrl+C
'\n'

BTW pthread_kill was only added to Python 3.3, so is not available for Python 2. I’m not sure what the best fix for 2.7 would be. Maybe it’s not worth it, or maybe you can find another way to a signal to the user process or its main thread without interfering with any background threads that Idle needs. But I don’t know enough about how Idle works to offer a solution.

A fix for 3.6+ will be better than nothing ;-)

Other operation which will block the process will do the same as time.sleep(), too.

    >>> import socket
    >>> s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    >>> s.bind(('', 25000))
    >>> s.recv(100)
    <Control-C>

IDLE will ignore this control-c, too.

Based on Martin's patch, I slightly changed the logic for the patch.

So, I add a `finish` flag to detect the code in Executive.runcode is done or not. `interrupt_the_server` will first SIGINT via interrupt_main, if this doesn't work after 0.2 seconds, it will then sent the signal by pthread_kill to main_thread.

This will work on Linux, and MacOS which I tested. And it will avoid the different behavior for `input()` (which I believe is because of the GNU readline been used.) and `time.sleep()` in only using `interrupt_main` or `signal.pthread_kill`.

On windows, s=input() is cleanly interrupted by ^-C, leaving s unbound, before and after the patch.  time.sleep(1) is not interrupted, before and after the patch.  Ditto for the socket test.  Louie, what test are you using on *nix?

It still appears that for Windows a patch to signalmodule.c is still needed.  Eryk and Nathaniel, can you agree on an actual patch?

Eryk: off topic a bit, but
>>> open('con', 'w').write('spam\n')
Traceback (most recent call last):
  File "<pyshell#3>", line 1, in <module>
    open('con', 'w').write('spam\n')
OSError: [WinError 6] The handle is invalid: 'con'

In terms of general design cleanliness, I think it would be better to make `interrupt_main` work reliably than to have IDLE add workarounds for its unreliability.

AFAICT the ideal, minimal redundancy solution would be:

- interrupt_main just calls raise(SIGINT)
- bpo-21895 is fixed by adding a few lines to Python's C-level signal handler on Unix-likes to check if it's running in the main thread, and if not then use pthread_kill to redirect the signal to the main thread.

If trying to fix bpo-28195 is too controversial, then a minimal change would be:

- in interrupt_main, if we're on Windows, call raise(SIGINT); otherwise, call pthread_kill(main_thread, SIGINT)

And in either case we'd also have to document that interrupt_main is now equivalent to the user having hit control-C. But AFAICT that's what all its users really want anyway.

Sorry, I meant bpo-21895.

> interrupt_main is now equivalent to the user having hit control-C. 

That's true on a POSIX system with pthread_kill, but not really for a Windows process that's attached to a console.      

A real Ctrl+C executes the registered control handlers for the process. To emulate this, PyErr_SetInterrupt could try calling GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0) to broadcast a Ctrl+C event. If the latter fails, it can fall back on raise(SIGINT), such as when the process isn't attached to a console.

One problem is that GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0) doesn't cancel a blocking console read like Ctrl+C does. Python's C handler could call CancelSynchronousIo(hMainThread) to address this problem in general. Unfortunately, when a console read is canceled in the client, it isn't canceled in the console itself. The contents of the read will be discarded, but it's a bit clunky that the user has to press enter.

Terry, I assume you ran IDLE via pyw.exe or pythonw.exe, which won't inherit the console of its parent. You have to use py.exe or python.exe. In this case you can also use sys.__stdout__.write('spam\n').

If you run via pythonw.exe or pyw.exe from a command prompt, you could use ctypes to attach to the console, but it would involve searching back to find the cmd.exe or powershell.exe PID to call AttachConsole(pid). More simply you can call ctypes.WinDLL('kernel32').AllocConsole() to get a new one.

It appears you are right about open('con'...).

However, this issue is about the fact that 
>>> import time; time.sleep(10)
^C
...
KeyboardInterrupt

works in the console but does not work in the IDLE Shell in default mode.  In default mode (-n not on the command line) entered code is executed in a separate process and and ^C causes a message to be sent to the separate process to simulate ^C keypress by calling 'something'.

The indirect process works for 'normal' code like
>>> s = input('Hi')
so input() is not part of this issue, except that it or its interruption should not be broken.  Ditto for
>>> while True: pass

> A real Ctrl+C executes the registered control handlers for the process.

Right, but it's *extremely* unusual for a python 3 program to have a control handler registered directly via SetConsoleCtrlHandler. This isn't an API that the interpreter uses or exposes. The vast majority of programs receive control-C notification by letting the C runtime convert the low level console event into a SIGINT, and then receiving this via the signal module.

> To emulate this, PyErr_SetInterrupt could try calling GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0) to broadcast a Ctrl+C event. 

But as mentioned up thread, this is really flaky - and even when it works it tends to kill random other processes, which is *certainly* not what anyone expects from calling interrupt_main. You can't really call it experimentally – even trying to call it can do stuff like cause appveyor tests to lock up.

Given these two issues, I think that emulating control-C at the signal level makes the best tradeoffs. Something that works 100% reliably for 99.99% of python programs is way better than something that is 80% reliable for 100% of programs.

> One problem is that GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0) doesn't cancel a blocking console read like Ctrl+C does. Python's C handler could call CancelSynchronousIo(hMainThread) to address this problem in general. Unfortunately, when a console read is canceled in the client, it isn't canceled in the console itself. The contents of the read will be discarded, but it's a bit clunky that the user has to press enter.

This might be something to address as a separate issue? I'm guessing this doesn't affect idle, and it doesn't affect time.sleep, which seem to be the main topics of this thread, plus it sounds like any solution would be mostly orthogonal.

+1 on having PyErr_SetInterrupt() call raise(SIGINT) or equivalent.

The strong claim that "interrupt_main is now equivalent to the user having hit control-C" is the reason I suggested broadcasting CTRL_C_EVENT via GenerateConsoleCtrlEvent. That operation isn't buggy on its own to my knowledge. There is a glitch due to the CRT. If a non-console process calls AllocConsole or AttachConsole, its list of registered handlers gets reset to the default handler that calls ExitProcess, and the CRT provides no way to set its handler again. 

I mentioned the possibility of calling CancelSynchronousIo in order to cancel a console read like Ctrl+C does (but clunkier) -- again, because of the equivalence claim. ERROR_OPERATION_ABORTED would need to be handled like EINTR on Unix. This would entail small changes to a lot of code, so it does need a separate issue if there's any support for this idea.

Then maybe simplest solution is to scale back the claim :-).

The important semantic change would be that right now, interrupt_main() is documented to cause KeyboardInterrupt to be raised in the main thread. So if you register a custom SIGINT handler that *doesn't* raise KeyboardInterrupt, interrupt_main() doesn't care: it doesn't invoke your custom handler, it just raises KeyboardInterrupt. My proposal would make it so that interrupt_main() does cause custom SIGINT handlers to be called, as well as triggering all of the interpreters existing machinery for prompt signal delivery (waking up time.sleep etc.), so in that respect it would be much more similar to a "real" control-C than it is now. This is a non-trivial semantic change, so it would need to be documented, but I'm not attached to the "equivalent to control-C" phrasing.

Even on UNIX, a "real" control-C is delivered to all processes in the foreground process group (or something like that, ttys are weird), but we would want interrupt_main() to only be delivered to the current process, so the equivalence wouldn't be exact there either.

> That operation isn't buggy on its own to my knowledge

I spent quite some time trying to make this work in order to test control-C handling in trio, and eventually gave up because I just could not make it work reliably. One notable effect is that if you call GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0) in an appveyor test run, then it breaks appveyor -- at the end of the run it freezes with "Terminate batch job (Y/N)?" and then sits until the test run times out. It's possible to work around this with some extremely obscure code (make sure you always spawn a subprocess with CREATE_NEW_CONSOLE or maybe CREATE_NEW_CONSOLE_GROUP etc.), but (a) we can't tell people that they need to do that before running code that uses interrupt_main(), and (b) even when I did that, then I still had bizarre issues I couldn't figure out, where sometimes the event just wouldn't be delivered unless I ran the interpreter with the -u option for unbuffered stdio (?!?!) (see [1]).

I just don't think this is something that CPython can use.

I eventually switched to simulating control-C events for testing by calling raise(SIGINT), and that's worked much much better.

> I mentioned the possibility of calling CancelSynchronousIo in order to cancel a console read like Ctrl+C does (but clunkier) -- again, because of the equivalence claim. ERROR_OPERATION_ABORTED would need to be handled like EINTR on Unix. This would entail small changes to a lot of code, so it does need a separate issue if there's any support for this idea.

It still wouldn't be equivalent because control-C only cancels console reads, but CancelSynchronousIo would cancel *any* blocking I/O operation that the main thread was doing, right? So e.g. if the main thread was blocked doing socket.recv, then control-C wouldn't interrupt that, but interrupt_main() / PyErr_SetInterrupt() would. I can certainly see an argument that Python's C-level signal handler *should* call CancelSynchronousIo in general, and then raise(SIGINT) and control-C would be equivalent because they both called CancelSynchronousIo, but yeah, that's way beyond the scope of this issue.

[1] https://github.com/python-trio/trio/commit/95843654173e3e826c34d70a90b369ba6edf2c23#diff-345cfb6c136028f9514b67ee7bb8e035R11

Please step back a bit and read the implementation of interrupt_main(): it calls PyErr_SetInterrupt() (in signalmodule.c!), which merely sets an internal flag saying SIGINT was received.

So, there: PyErr_SetInterrupt() already behaves like SIGINT, *except* that it doesn't actually deliver a C signal, it merely sets a flag.  Which is the reason that it fails waking up C syscalls like select().

Demonstration:

>>> def handler(signum, frame):
...     print("got signal %d!" % (signum,))
... 
>>> signal.signal(signal.SIGINT, handler)
<built-in function default_int_handler>
>>> _thread.interrupt_main()
got signal 2!


In the end, making interrupt_main() *actually* deliver a SIGINT instead of merely setting the internal flag for it will certainly be true to the original intent.

Note that, since interrupt_main() claims to interrupt the main thread, it would be ok to use pthread_kill() to make sure it's indeed the C syscall running in the main thread that gets interruped.

Also note that:

- PyErr_SetInterrupt() has been setting the SIGINT since the beginning in 1995 (changeset 06d511dd by Guido)
- interrupt_main() has been calling PyErr_SetInterrupt() since the beginning in 2003 (changeset a11e8461 by Kurt B Kaiser)

so it stands reasonable to keep the SIGINT behaviour.  The documentation is simply wrong and/or imprecise.

Github links to the aforementioned changesets:

"added PyErr_SetInterrupt(); NT ifdefs"
https://github.com/python/cpython/commit/06d511ddf5fe16468a3abd53344fa283b9981d73

"Add interrupt_main() to thread module."
https://github.com/python/cpython/commit/a11e84613579e2487bcb3967d3a2edbd0665343a

I doubt the Gnu Readline library nor Python’s readline module are relevant. The input function only uses Readline if sys.stdin is the original stdin terminal; I suspect Idle monkey-patches sys.stdin. The readline method reads directly from the file object; it never uses the Readline library (despite the unfortunate clash in names :).

Louie seems to have added two workarounds: the finish flag, and using interrupt_main with a timeout. Both seem racy. What if the flag is altered in another thread between when the flag is checked and when it is acted upon? What has to happen in the 0.2 s between calling interrupt_main and raising SIGINT?