Message295040
Thomas, thanks for the heads up. I would suggest something like the following patch to multiprocessing.Pool:
$ git diff
diff --git a/Lib/multiprocessing/queues.py b/Lib/multiprocessing/queues.py
index 7f77837..ebbb360 100644
--- a/Lib/multiprocessing/queues.py
+++ b/Lib/multiprocessing/queues.py
@@ -260,8 +260,16 @@ class Queue(object):
info('error in queue thread: %s', e)
return
else:
- import traceback
- traceback.print_exc()
+ self._on_queue_thread_error(e)
+
+ def _on_queue_thread_error(self, e):
+ """
+ Private API called when feeding data in the background thread
+ raises an exception. For overriding by concurrent.futures.
+ """
+ import traceback
+ traceback.print_exc()
+
_sentinel = object()
Then you can write your own Queue subclass in concurrent.futures to handle that error and clean up/restart whatever needs to be cleaned up or restarted. What do you think? |
|
Date |
User |
Action |
Args |
2017-06-02 16:52:01 | pitrou | set | recipients:
+ pitrou, davin, xiang.zhang, tomMoral, grzgrzgrz3 |
2017-06-02 16:52:01 | pitrou | set | messageid: <1496422321.02.0.740107834517.issue30414@psf.upfronthosting.co.za> |
2017-06-02 16:52:01 | pitrou | link | issue30414 messages |
2017-06-02 16:52:00 | pitrou | create | |
|