classification
Title: multiprocessing JoinableQueue's join function with timeout
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.4
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: legordian, sbt, trent, tumert, zach.ware
Priority: normal Keywords: patch

Created on 2012-09-20 13:29 by legordian, last changed 2014-06-22 20:31 by tumert.

Files
File name Uploaded Description Edit
JoinableQueue_with_timeout.patch legordian, 2012-09-20 13:29 Patch adding a timeout to multiprocessing.JoinableQueue.join() review
Messages (4)
msg170812 - (view) Author: Karl Bicker (legordian) Date: 2012-09-20 13:29
The multiprocessing.JoinableQueue's function join() should have a timeout argument so that one can check on other things while waiting for a queue to finish.

As join() uses a condition to wait anyway, a timeout is easily implemented and passed to the Condidition.wait(). Patch is attached.
msg170894 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2012-09-21 14:39
I notice that queue.Queue.join() does not have a timeout parameter either.

Have you hit a particular problem that would be substantially easier with the patch?
msg170895 - (view) Author: Karl Bicker (legordian) Date: 2012-09-21 15:11
Actually, I would like to check if the threads working on the queue are still alive. 

However, it occurs to me now that I would need a facility to distinguish between a timeout and an actual join. Unfortunately, my original patch does not provide this, one would have to implement an additional check and think about the correct way to signal a timeout.

Also, there seem to be other people looking for this feature: http://stackoverflow.com/questions/1564501/add-timeout-argument-to-pythons-queue-join
msg221307 - (view) Author: Tumer Topcu (tumert) Date: 2014-06-22 20:30
Another option is instead of adding a timeout parameter, a new function to check if everything is done. Like:

def all_tasks_finished(self)
    return self._unfinished_tasks._semlock._is_zero()

Which can then be utilized exactly shown in the above noted stackoverflow answer. This way it will be consistent with queue.py
History
Date User Action Args
2014-06-22 20:31:07tumertsetnosy: + trent
2014-06-22 20:30:53tumertsetmessages: + msg221307
2014-06-22 19:02:19tumertsetnosy: + zach.ware, tumert
2012-09-21 15:11:14legordiansetmessages: + msg170895
2012-09-21 14:39:31sbtsetmessages: + msg170894
versions: - Python 2.7, Python 3.2, Python 3.3
2012-09-20 15:06:35sbtsetnosy: + sbt
2012-09-20 13:40:32berker.peksagsetversions: - Python 3.1
2012-09-20 13:29:52legordiancreate