Message109585
I have recently begun using multiprocessing for a variety of batch
jobs. It's a great library, and it's been quite useful. However, I have been bitten several times by situations where a worker process in a Pool will unexpectedly die, leaving multiprocessing hanging in a wait. A simple example of this is produced by the following:
"""
#!/usr/bin/env python
import multiprocessing, sys
def foo(x):
sys.exit(1)
multiprocessing.Pool(1).apply(foo, [1])
"""
The child will exit and the parent will hang forever. A similar occurrence happens if one pushes C-c while a child process is running (this special case is noted in http://bugs.python.org/issue8296) or killed by a signal.
Attached is a patch to handle unexpected terminations of children
processes and prevent the parent process from hanging. A test case is included. (Developed and tested on 64-bit Ubuntu.) Please let me know what you think. Thanks! |
|
Date |
User |
Action |
Args |
2010-07-08 20:00:51 | gdb | set | recipients:
+ gdb |
2010-07-08 20:00:51 | gdb | set | messageid: <1278619251.36.0.833806205056.issue9205@psf.upfronthosting.co.za> |
2010-07-08 20:00:44 | gdb | link | issue9205 messages |
2010-07-08 20:00:44 | gdb | create | |
|