This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Implement a CSP-style channel
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: anacrolix, gvanrossum, loewis, snim2
Priority: normal Keywords:

Created on 2012-02-20 00:56 by anacrolix, last changed 2022-04-11 14:57 by admin.

Messages (6)
msg153748 - (view) Author: Matt Joiner (anacrolix) Date: 2012-02-20 00:56
From the mailing list, there is some interest in a CSP-style channel. http://mail.python.org/pipermail/python-ideas/2012-February/014073.html
msg153762 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2012-02-20 04:06
Note that Go channels have an optional buffer.  That makes them more like the queue module, except the latter doesn't support zero-sized queues.  How hard would it be to add that?  And are there really use cases in Python for unbuffered channels?
msg153808 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-02-20 21:12
I think zero-sized queues are exactly the right answer here.
msg153856 - (view) Author: Matt Joiner (anacrolix) Date: 2012-02-21 07:47
As I see it, here are the desirable features of CSP-style concurrency as it pertains to channels:

1) At least an unbuffered mode
2) Can be marked closed
3) Block on multiple send/receives until one can proceed

Specifically features 1 and 2 could be bolted onto queue.Queue with excellent backward compatibility with existing usage of Queue. As mentioned on the mailing list, maxsize=None would mean infinite queue, and maxsize=0 would become the unbuffered mode Currently maxsize<=0 means infinite. Existing code that assumed one or the other and explicitly created Queues this way would have to change (I'd suspect almost no code would be affected).

Feature 3 allows for all the awesomeness of CSP channels, but I think it requires a redesign under the covers of Queue to allow waiters to be woken from arbitrary Queues to which they are subscribed. The synchronization internals are quite complex.

There appears to be several ways to proceed:

1) Add unbuffered mode, closing, and other syntactical sugar to queue.Queue.
2) Do (1) and additionally rework the internals to allow blocking on on several Queue actions at once.
3) Add an unbuffered-only, closable threading.Channel for use with higher level primitives (as Guido suggested).
4) Make no changes to queue, and create a brand-new module with full blown CSP channels.
msg153886 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2012-02-21 17:55
I'd like to veto changing the meaning of 0, it's going to break somebody's code. It's easy enough to add some other keyword argument to request an unbuffered queue.  Other than that, (1) and (2) seem fine to me.  Please consider making these changes for multiprocessing as well.

For (3) I like the idea, but I think we should be more ambitious and instead add a way to wait for multiple locks (or other things) across the board, not just in the queue module.  (Maybe integrating with I/O waits as well, though that feels too ambitious, and is likely to stifle any attempts at a cross-platform implementation.)
msg221703 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-27 18:25
@Matt are you interested in following up on this?
History
Date User Action Args
2022-04-11 14:57:26adminsetgithub: 58268
2019-03-15 23:56:34BreamoreBoysetnosy: - BreamoreBoy
2014-06-27 18:25:24BreamoreBoysetnosy: + BreamoreBoy

messages: + msg221703
versions: + Python 3.5, - Python 3.3
2014-04-03 22:04:56snim2setnosy: + snim2
2012-02-21 17:55:01gvanrossumsetmessages: + msg153886
2012-02-21 07:47:45anacrolixsetmessages: + msg153856
2012-02-20 21:12:21loewissetnosy: + loewis
messages: + msg153808
2012-02-20 04:06:42gvanrossumsetmessages: + msg153762
2012-02-20 04:01:28gvanrossumsetnosy: + gvanrossum
2012-02-20 00:56:06anacrolixcreate