classification
Title: test_multiprocessing stuck in test_make_pool if run in a loop
Type: Stage:
Components: Tests Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: jnoller, neologix, sandro.tosi
Priority: normal Keywords:

Created on 2010-10-16 12:39 by sandro.tosi, last changed 2011-04-14 16:21 by sandro.tosi. This issue is now closed.

Messages (4)
msg118867 - (view) Author: Sandro Tosi (sandro.tosi) * (Python committer) Date: 2010-10-16 12:39
Hello,
when trying to see if issue6661 was still valid, I run test_multiprocessing in a tight loop and I got it reproducibly stuck on test_make_pool:

...

test_imap_unordered (test.test_multiprocessing.WithManagerTestPool) ... ok
test_make_pool (test.test_multiprocessing.WithManagerTestPool) ... 

when I enter Ctrl+C here's the traceback (on 2.7 HEAD):

test_make_pool (test.test_multiprocessing.WithManagerTestPool) ... ^CProcess PoolWorker-5:3:
Traceback (most recent call last):
  File "/home/morph/python-dev/release2.7-maint/Lib/multiprocessing/process.py", line 232, in _bootstrap
Process PoolWorker-5:2:
Traceback (most recent call last):
Process PoolWorker-5:4:
  File "/home/morph/python-dev/release2.7-maint/Lib/multiprocessing/process.py", line 232, in _bootstrap
Traceback (most recent call last):
  File "/home/morph/python-dev/release2.7-maint/Lib/multiprocessing/process.py", line 232, in _bootstrap
Process PoolWorker-5:1:
Traceback (most recent call last):
  File "/home/morph/python-dev/release2.7-maint/Lib/multiprocessing/process.py", line 232, in _bootstrap
Process PoolWorker-79:
Process PoolWorker-80:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/morph/python-dev/release2.7-maint/Lib/multiprocessing/process.py", line 232, in _bootstrap
  File "/home/morph/python-dev/release2.7-maint/Lib/multiprocessing/process.py", line 232, in _bootstrap
    self.run()
    self.run()
  File "/home/morph/python-dev/release2.7-maint/Lib/multiprocessing/process.py", line 88, in run
  File "/home/morph/python-dev/release2.7-maint/Lib/multiprocessing/process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
  File "/home/morph/python-dev/release2.7-maint/Lib/multiprocessing/pool.py", line 59, in worker
  File "/home/morph/python-dev/release2.7-maint/Lib/multiprocessing/pool.py", line 59, in worker
    task = get()
    task = get()
  File "/home/morph/python-dev/release2.7-maint/Lib/multiprocessing/queues.py", line 352, in get
  File "/home/morph/python-dev/release2.7-maint/Lib/multiprocessing/queues.py", line 350, in get

Test suite interrupted by signal SIGINT.
1 test omitted:
    test_multiprocessing

and on py3k:

test_make_pool (test.test_multiprocessing.WithProcessesTestPool) ... ^CProcess PoolWorker-5:1:
Process PoolWorker-3:
Process PoolWorker-5:3:
Process PoolWorker-5:4:
Process PoolWorker-5:2:
Process PoolWorker-4:
Process PoolWorker-2:
Process PoolWorker-1:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 233, in _bootstrap
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 233, in _bootstrap
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 233, in _bootstrap
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 233, in _bootstrap
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 233, in _bootstrap
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 233, in _bootstrap
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 233, in _bootstrap
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 233, in _bootstrap
Process PoolWorker-29:
Traceback (most recent call last):
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 233, in _bootstrap

Test suite interrupted by signal SIGINT.
1 test omitted:
    self.run()
    self.run()
    self.run()
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 88, in run
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 88, in run
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/pool.py", line 59, in worker
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/pool.py", line 59, in worker
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/pool.py", line 59, in worker
    self.run()
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/pool.py", line 59, in worker
    self.run()
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 88, in run
    self.run()
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/pool.py", line 59, in worker
    self._target(*self._args, **self._kwargs)
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/pool.py", line 59, in worker
    self.run()
    self.run()
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 88, in run
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/pool.py", line 59, in worker
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/pool.py", line 59, in worker
    task = get()
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/queues.py", line 350, in get
    task = get()
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/queues.py", line 350, in get
    racquire()
KeyboardInterrupt
    racquire()
KeyboardInterrupt
    task = get()
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/queues.py", line 352, in get
    task = get()
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/queues.py", line 350, in get
    return recv()
    racquire()
KeyboardInterrupt
KeyboardInterrupt
    task = get()
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/queues.py", line 350, in get
    racquire()
    task = get()
    task = get()
KeyboardInterrupt
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/queues.py", line 350, in get
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/queues.py", line 350, in get
    racquire()
KeyboardInterrupt
    racquire()
KeyboardInterrupt
    task = get()
  File "/home/morph/python-dev/py3k/Lib/multiprocessing/queues.py", line 352, in get
    return recv()
KeyboardInterrupt
    test_multiprocessing

It's reproducible here so I can make any tests you want to verify the problem (I don't know what to look at, sorry); just to give some more info, here's a Debian sid on AMD64.

Cheers,
Sandro

PS: it's my first bug report to python, so please be gentle and forgive any mistakes I could have made :)
msg119228 - (view) Author: Sandro Tosi (sandro.tosi) * (Python committer) Date: 2010-10-20 17:44
just for the reference, the loop I ran was:

 while date ; do ./python Lib/test/regrtest.py -v test_multiprocessing ; done
msg133657 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-04-13 09:25
It's probably a duplicate of http://bugs.python.org/issue8428
It would be nice if you could try to reproduce it with a py3k snapshot though, just to be sure.
msg133746 - (view) Author: Sandro Tosi (sandro.tosi) * (Python committer) Date: 2011-04-14 16:21
Thanks for reminding me of this issue. I let the same tight loop run on an up-to-date python for a whole day and not stuck happened.
History
Date User Action Args
2011-04-14 16:21:21sandro.tosisetstatus: open -> closed
resolution: out of date
messages: + msg133746
2011-04-13 09:25:25neologixsetnosy: + neologix
messages: + msg133657
2010-10-20 17:44:09sandro.tosisetmessages: + msg119228
2010-10-16 12:39:31sandro.tosicreate