This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author merrellb
Recipients merrellb
Date 2008-12-14.16:48:01
SpamBayes Score 1.6300037e-05
Marked as misclassified No
Message-id <1229273282.85.0.657676661464.issue4660@psf.upfronthosting.co.za>
In-reply-to
Content
Despite carefully matching my get() and task_done() statements I would 
often trigger "raise ValueError('task_done() called too many times')" in 
my multiprocessing.JoinableQueue (multiprocessing/queues.py)

Looking over the code (and a lot of debug logging), it appears that the 
issue arises from JoinableQueue.put() not being protected with a locking 
mechanism.  A preemption after the first line allows other processes to 
resume without releasing the _unfinished_tasks semaphore.

The simplest solution seems to be allowing task_done() to block while 
waiting to acquire the _unfinished_tasks semaphore.

Replacing:
if not self._unfinished_tasks.acquire(False):
  raise ValueError('task_done() called too many times')

With simply:
self._unfinished_tasks.acquire()

This would however remove the error checking provided (given the many 
far more subtler error that can be made, I might argue it is of limited 
value).  Alternately the JoinableQueue.put() method could be better 
protected.
History
Date User Action Args
2008-12-14 16:48:02merrellbsetrecipients: + merrellb
2008-12-14 16:48:02merrellbsetmessageid: <1229273282.85.0.657676661464.issue4660@psf.upfronthosting.co.za>
2008-12-14 16:48:02merrellblinkissue4660 messages
2008-12-14 16:48:01merrellbcreate