This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: queue.Queue.put should acquire mutex for unfinished_tasks
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.2
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: patrick.vrijlandt, rhettinger
Priority: normal Keywords:

Created on 2011-05-03 19:30 by patrick.vrijlandt, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
unnamed patrick.vrijlandt, 2011-05-04 06:09
Messages (3)
msg135063 - (view) Author: patrick vrijlandt (patrick.vrijlandt) Date: 2011-05-03 19:30
Line 154 in standard library's queue.py, in the definition of Queue.put() is: 
self.unfinished_tasks += 1

This line should be protected by acquiring the all_tasks_done lock.
Theoretically, the increment could occur somewhere during task_done()!

Additional note:
Personally, I would like the increment to occur *before* instead of *after* _put(). This is because I would like to be able to implement a _put() that does not put everything. This put should be able to undo the increment. As it is, calling task_done() from put might decrease the counter to zero inadvertently.
msg135064 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-05-03 19:46
> This line should be protected by acquiring the all_tasks_done lock.

All of three of the condition variables share the same underlying lock.  The increment occurs only with the lock has been acquired. 

> Theoretically, the increment could occur somewhere during task_done()!

I don't understand what you mean.  The semantics of task_done() method implies that the count gets decremented.

> Personally, I would like the increment to occur *before* 
> instead of *after* _put().

There may be some merit to this but I don't see how it matters much since both occur within the context of the same lock being held.  A user defined method can still add or subtract any number it wants from the unfinished task count.  The result of +1 -5 is the same as -5 +1.

I'm reluctant to change the order though because the code is already published and some user's _put code may be inspecting the unfinished tasks count.  I wouldn't want to break that code without good reason.
msg135097 - (view) Author: patrick vrijlandt (patrick.vrijlandt) Date: 2011-05-04 06:09
I agree. Please close the ticket.

Thanks,

Patrick

2011/5/3 Raymond Hettinger <report@bugs.python.org>

>
> Raymond Hettinger <raymond.hettinger@gmail.com> added the comment:
>
> > This line should be protected by acquiring the all_tasks_done lock.
>
> All of three of the condition variables share the same underlying lock.
>  The increment occurs only with the lock has been acquired.
>
> > Theoretically, the increment could occur somewhere during task_done()!
>
> I don't understand what you mean.  The semantics of task_done() method
> implies that the count gets decremented.
>
> > Personally, I would like the increment to occur *before*
> > instead of *after* _put().
>
> There may be some merit to this but I don't see how it matters much since
> both occur within the context of the same lock being held.  A user defined
> method can still add or subtract any number it wants from the unfinished
> task count.  The result of +1 -5 is the same as -5 +1.
>
> I'm reluctant to change the order though because the code is already
> published and some user's _put code may be inspecting the unfinished tasks
> count.  I wouldn't want to break that code without good reason.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue11987>
> _______________________________________
>
History
Date User Action Args
2022-04-11 14:57:16adminsetgithub: 56196
2011-05-04 06:14:09rhettingersetstatus: open -> closed
resolution: not a bug
2011-05-04 06:09:22patrick.vrijlandtsetfiles: + unnamed

messages: + msg135097
2011-05-03 19:46:42rhettingersetmessages: + msg135064
2011-05-03 19:31:07rhettingersetassignee: rhettinger

nosy: + rhettinger
2011-05-03 19:30:08patrick.vrijlandtcreate