Author OG7
Recipients LambertDW, OG7, jnoller, zzzeek
Date 2009-06-10.08:58:08
SpamBayes Score 8.3029e-11
Marked as misclassified No
Message-id <1244624290.28.0.38069051667.issue5331@psf.upfronthosting.co.za>
In-reply-to
Content
Apparently the pool workers die all at once, just after the pool creates
them and before the pool is used.
I added a few lines to multiprocessing/pool.py to get the stack and the
exception backtrace.

        except (EOFError, IOError):
            import traceback
            debug(traceback.format_exc())
            debug(''.join(traceback.format_stack()))
            debug('worker got EOFError or IOError -- exiting')
            break


INFO::Rule dispatcher::multiprocessing::child process calling self.run()
DEBUG::Rule dispatcher::multiprocessing::created semlock with handle
3082559488
DEBUG::Rule dispatcher::multiprocessing::created semlock with handle
3082104832
DEBUG::Rule dispatcher::multiprocessing::created semlock with handle
3081826304
DEBUG::Rule dispatcher::multiprocessing::created semlock with handle
3081822208
INFO::PoolWorker-3:1::multiprocessing::child process calling self.run()
DEBUG::PoolWorker-3:1::multiprocessing::Traceback (most recent call last):
  File
"/home/who/.buildout/eggs/multiprocessing-2.6.1.1-py2.6-linux-i686.egg/multiprocessing/pool.py",
line 57, in worker
    task = get()
  File
"/home/who/.buildout/eggs/multiprocessing-2.6.1.1-py2.6-linux-i686.egg/multiprocessing/queues.py",
line 339, in get
    return recv()
IOError: [Errno 9] Bad file descriptor

DEBUG::PoolWorker-3:1::multiprocessing::  File
"/home/who/var/co/git-svn/what-base/correlator/bin/correlator", line 30,
in <module>
    what.corr.actors.main.main()
  File
"/home/who/var/co/git-svn/what-base/correlator/lib/what/corr/actors/main.py",
line 47, in main
    rrp.start()
  File
"/home/who/.buildout/eggs/multiprocessing-2.6.1.1-py2.6-linux-i686.egg/multiprocessing/process.py",
line 109, in start
    self._popen = Popen(self)
  File
"/home/who/.buildout/eggs/multiprocessing-2.6.1.1-py2.6-linux-i686.egg/multiprocessing/forking.py",
line 99, in __init__
    code = process_obj._bootstrap()
  File
"/home/who/.buildout/eggs/multiprocessing-2.6.1.1-py2.6-linux-i686.egg/multiprocessing/process.py",
line 236, in _bootstrap
    self.run()
  File
"/home/who/.buildout/eggs/multiprocessing-2.6.1.1-py2.6-linux-i686.egg/multiprocessing/process.py",
line 93, in run
    self._target(*self._args, **self._kwargs)
  File
"/home/who/var/co/git-svn/what-base/correlator/lib/what/corr/actors/rule_dispatcher.py",
line 26, in main
    initargs=(manager.agg_msgs_queue, ))
  File
"/home/who/.buildout/eggs/multiprocessing-2.6.1.1-py2.6-linux-i686.egg/multiprocessing/__init__.py",
line 232, in Pool
    return Pool(processes, initializer, initargs)
  File
"/home/who/.buildout/eggs/multiprocessing-2.6.1.1-py2.6-linux-i686.egg/multiprocessing/pool.py",
line 107, in __init__
    w.start()
  File
"/home/who/.buildout/eggs/multiprocessing-2.6.1.1-py2.6-linux-i686.egg/multiprocessing/process.py",
line 109, in start
    self._popen = Popen(self)
  File
"/home/who/.buildout/eggs/multiprocessing-2.6.1.1-py2.6-linux-i686.egg/multiprocessing/forking.py",
line 99, in __init__
    code = process_obj._bootstrap()
  File
"/home/who/.buildout/eggs/multiprocessing-2.6.1.1-py2.6-linux-i686.egg/multiprocessing/process.py",
line 236, in _bootstrap
    self.run()
  File
"/home/who/.buildout/eggs/multiprocessing-2.6.1.1-py2.6-linux-i686.egg/multiprocessing/process.py",
line 93, in run
    self._target(*self._args, **self._kwargs)
  File
"/home/who/.buildout/eggs/multiprocessing-2.6.1.1-py2.6-linux-i686.egg/multiprocessing/pool.py",
line 61, in worker
    debug(''.join(traceback.format_stack()))

DEBUG::PoolWorker-3:1::multiprocessing::worker got EOFError or IOError
-- exiting
INFO::PoolWorker-3:1::multiprocessing::process shutting down
DEBUG::PoolWorker-3:1::multiprocessing::running all "atexit" finalizers
with priority >= 0
History
Date User Action Args
2009-06-10 08:58:10OG7setrecipients: + OG7, LambertDW, jnoller, zzzeek
2009-06-10 08:58:10OG7setmessageid: <1244624290.28.0.38069051667.issue5331@psf.upfronthosting.co.za>
2009-06-10 08:58:09OG7linkissue5331 messages
2009-06-10 08:58:08OG7create