This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: allow deque to act as a thread-safe circular buffer
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: chris.jerdonek, rhettinger, s7v7nislands@gmail.com
Priority: low Keywords:

Created on 2012-07-11 22:20 by chris.jerdonek, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (6)
msg165276 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2012-07-11 22:20
It seems like it would be useful if collections.deque had a thread-safe method that could rotate(1) and return the rotated value.

This would let deque to act as a thread-safe circular buffer (e.g. if serving jobs to multiple threads in a loop, like `python -m test --forever`).
msg165279 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2012-07-12 05:27
By thread-safe you mean that the operation should be atomic?
msg165280 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2012-07-12 05:37
Yes, atomic. I was under the impression that the existing deque.rotate() is atomic, in which case deque.rotate(1) almost provides what I'm suggesting (lacking only the return value).
msg165281 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2012-07-12 05:47
We make almost no guarantees about atomicity.  If something in CPython happens to be atomic, it is not guaranteed to hold in other implementations.   The recommendation is to use locks anywhere you need a critical section of code.

FWIW, the term thread-safe is taken to mean, "won't break in a multi-threaded environment".  That is a bit different from atomic.  For example, the decimal module was designed with thread local contexts, but there are no atomic decimal operations.
msg165282 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2012-07-12 06:04
Thanks for the info.  A couple questions: what does "won't break" mean -- that it won't throw an exception of a type that it wouldn't normally throw in a single-threaded environment?  And does this mean that not even deque.pop() is atomic?
msg165315 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2012-07-12 15:58
> A couple questions: what does "won't break" mean 

That means that its internal invariants always survive in a multi-threaded environment.

> that it won't throw an exception of a type that it 
> wouldn't normally throw in a single-threaded environment? 

Unless that exception is being raised to indicate that a resource is in  use (much that same as a database notifying you that it can't make a change because the DB is currently locked).
History
Date User Action Args
2022-04-11 14:57:32adminsetgithub: 59535
2015-04-25 15:10:22s7v7nislands@gmail.comsetnosy: + s7v7nislands@gmail.com
2012-07-12 15:58:38rhettingersetmessages: + msg165315
2012-07-12 06:04:48chris.jerdoneksetmessages: + msg165282
2012-07-12 05:47:00rhettingersetstatus: open -> closed
resolution: rejected
messages: + msg165281
2012-07-12 05:37:56chris.jerdoneksetmessages: + msg165280
2012-07-12 05:27:02rhettingersetmessages: + msg165279
2012-07-12 05:23:32rhettingersetpriority: normal -> low
assignee: rhettinger
versions: + Python 3.4
2012-07-11 22:20:37chris.jerdonekcreate