This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author merrellb
Recipients merrellb
Date 2008-12-14.16:48:01
SpamBayes Score 1.63e-05
Marked as misclassified No
Message-id <>
Despite carefully matching my get() and task_done() statements I would 
often trigger "raise ValueError('task_done() called too many times')" in 
my multiprocessing.JoinableQueue (multiprocessing/

Looking over the code (and a lot of debug logging), it appears that the 
issue arises from JoinableQueue.put() not being protected with a locking 
mechanism.  A preemption after the first line allows other processes to 
resume without releasing the _unfinished_tasks semaphore.

The simplest solution seems to be allowing task_done() to block while 
waiting to acquire the _unfinished_tasks semaphore.

if not self._unfinished_tasks.acquire(False):
  raise ValueError('task_done() called too many times')

With simply:

This would however remove the error checking provided (given the many 
far more subtler error that can be made, I might argue it is of limited 
value).  Alternately the JoinableQueue.put() method could be better 
Date User Action Args
2008-12-14 16:48:02merrellbsetrecipients: + merrellb
2008-12-14 16:48:02merrellbsetmessageid: <>
2008-12-14 16:48:02merrellblinkissue4660 messages
2008-12-14 16:48:01merrellbcreate