classification
Title: Asynchronous exceptions between threads
Type: enhancement Stage: test needed
Components: Extension Modules Versions: Python 3.2
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, akuchling, loewis, phr, pitrou, tim.peters
Priority: normal Keywords:

Created on 2002-01-11 08:46 by phr, last changed 2010-08-18 14:21 by akuchling. This issue is now closed.

Files
File name Uploaded Description Edit
thr.py loewis, 2002-01-11 16:19
Messages (13)
msg53433 - (view) Author: paul rubin (phr) Date: 2002-01-11 08:46
I may be missing something but there doesn't seem to
be any easy, correct way to exit a Python application
and shut down the interpreter.  sys.exit() raises
the SysExit exception in the current thread but other
threads keep running, so the application doesn't stop.
You can do something brutal like os.kill() your
process, but then cleanup actions (finally clauses)
don't get run.  Or you can create some elaborate
application-specific framework for propagating an
exit flag from one thread to all the rest.  That's
incredibly lame.  

What's needed is a simple function (maybe sys.allexit)
that raises SysExit in all the threads of the
application (it might have to first stop all the 
threads using the GIL or something like that).  

I'm surprised there isn't already something like this, 
but there's been some c.l.py discussion about it and 
it really seems to not be there.  Is there some 
problem with the idea?  I don't see one but I can't be 
sure.
msg53434 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2002-01-11 15:46
Logged In: YES 
user_id=6380

I'm struggling with this in a threaded distributed app we're
developing for Zope Corp, so I agree that something like
this would be nice. I think for starters, there should be a
way to raise an asynchronous exception in a specific thread;
all threads is easy then.

There are plenty of implementation problems with this,
however: if a thread is blocked in I/O, there's no portable
way to get it to bail out of that I/O operation.

There are also plenty of application-level issues: if
asynchronous exceptions are common, maybe it is necessary to
provide a way to temporarily block such exceptions in order
to provide safety in some situations.

Etc., etc. So please spare us your surprise and help looking
for a mechanism that can be implemented.
msg53435 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2002-01-11 16:19
Logged In: YES 
user_id=21627

Can you produce an example demonstrating the problem? Please
see the attached thr.py. It terminates fine; I fail to see
the problem.

When SystemExit is raised, it will eventually invoke the C
library's exit(). If that is not incredibly lame, it will
terminate all threads.
msg53436 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2002-01-11 21:16
Logged In: YES 
user_id=31435

Martin, change your program to do sys.exit(1) inside func() 
on the first iteration (or raise SystemExit similarly), and 
whether you're on Windows or Linux you should find that 
while the func() thread goes away, the main program keeps 
on running.  If you do os._exit(1) instead, on Windows the 
main program does go away, but Guido said it doesn't on 
Linux (which didn't surprise me, given Linux's conflation 
of threads with processes).  See also the "Killing Threads" 
thread in Python-Dev from late May of 2001, which said all 
the same things.

The only thing I may <wink> have learned since then is that 
supposedly the *C* exit() function terminates all threads 
on Linux, via an atexit handler installed by libc, and so 
_exit() on Linux doesn't terminate all threads because _exit
() doesn't call atexit functions.
msg53437 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2002-01-12 01:05
Logged In: YES 
user_id=21627

Tim, I'm not surprised that raising SystemExit in a thread
fails to exit Python; the reason simply is that
threadmodule.c:t_bootstrap choses to ignore
PyExc_SystemExit, when it instead should invoke
PyErr_PrintEx instead.

The bug is obviously that exit_thread also raises
SystemExit, when people apparently really want sys.exit and
thread.exit to be two different things. So I think
thread.exit should raise ThreadExit, and raising SystemExit
in a thread should cause Py_Exit to be invoked (along with
all other necessary cleanup). In short, 2.12 of
threadmodule.c was wrong, IMO.
msg53438 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2002-01-12 20:03
Logged In: YES 
user_id=6380

(1) It's too late to change the meaning of SystemExit --
people are relying on SystemExit exiting the thread only.

(2) If we provide a way for a thread to exit the whole
program, how will finally clauses in other threads
(including the main thread) be executed?
msg53439 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2002-01-14 08:40
Logged In: YES 
user_id=21627

On (1): This is quite unfortunate, as I do think sys.exit
and thread.exit should do different things. There would be
approaches of further subclassing SystemExit; preserving the
property that raising SystemExit in a thread exits the
thread only - would such a solution be acceptable?

On (2): This is no different from raising SystemExit in the
main thread, which also does not invoke finally clauses in
other threads.

I don't think there can be a complete, reliable
implementation of thread cancellation, only a best-effort
approach. For POSIX, I'd suggest sending SIGUSR2 to each
thread through pthread_kill. 

We might need to add a mechanism indicating whether a thread
is ready to be cancelled, similar to the POSIX cancellation
points. That would prevent the signal from arriving in an
arbitrary library function, and defer thread termination
until the library function completes. For blocking system
calls, sending signals would need to be activated
explicitly, to allow aborting them.
msg53440 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2002-01-15 19:33
Logged In: YES 
user_id=6380

(1) I think subclassing SystemExit can work.

(2) But the difference is that killing the main thread
without giving it a chance to clean up is worse than killing
another thread.

Does anybody want to write a PEP on thread cancellation? It
sure looks like a hairy issue!
msg53441 - (view) Author: paul rubin (phr) Date: 2002-01-15 19:40
Logged In: YES 
user_id=72053

I don't feel qualified to write such a PEP.  You guys are 
more on top of this than I am.  Maybe there's no portable 
way to do asynchronous exceptions between threads.  I don't 
know what happens if you hit ctrl-C when running a multi-
threaded Linux program.  I don't think ctrl-C even works in 
Windows--all you can really do is bring up the task manager 
and blow away the whole process.  But I think asynchronous 
exceptions are worth having even if they're OS specific and 
only work on some systems.

As for how a thread can exit the whole program, I thought 
Guido's first message (saying raise some asynchronous 
exception in all the threads) sounded fine, if it's 
feasible.
msg53442 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2006-12-21 14:52
Changing summary to be clearer; reclassifying as 'Feature Request' type.
msg114141 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-08-17 18:21
Is anyone likely to work on this given msg53440 quote "Does anybody want to write a PEP on thread cancellation? It sure looks like a hairy issue!"
msg114163 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-08-17 20:34
For the record, there is now a C API function (PyThreadState_SetAsyncExc()) to raise an asynchronous exception on a Python thread, although the signature is bizarre (it takes the thread id rather than the thread state structure itself).
msg114223 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2010-08-18 14:21
I'll close this issue, then.  Maybe something fancier needs to be built atop the AsyncExc() function to allow a single thread to terminate all other threads, but unless someone actually presents a current use case (or a PEP), there seems little to do.  Please re-open if there *is* something concrete to do.
History
Date User Action Args
2010-08-18 14:21:15akuchlingsetstatus: open -> closed
resolution: out of date
messages: + msg114223
2010-08-17 23:57:54gvanrossumsetnosy: - gvanrossum
2010-08-17 20:34:21pitrousetnosy: + pitrou
messages: + msg114163
2010-08-17 18:21:18BreamoreBoysetnosy: + BreamoreBoy

messages: + msg114141
versions: - Python 2.7
2009-05-15 03:05:37ajaksu2setstage: test needed
versions: + Python 2.7, Python 3.2, - Python 2.6
2008-01-06 12:10:58christian.heimessetversions: + Python 2.6
2002-01-11 08:46:40phrcreate