This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Add asyncio.Queue __aiter__, __anext__ methods
Type: enhancement Stage: resolved
Components: asyncio Versions: Python 3.7
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: Matt Rasband, RekGRpth, chris.jerdonek, gvanrossum, yselivanov
Priority: normal Keywords:

Created on 2016-11-23 04:58 by RekGRpth, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (9)
msg281536 - (view) Author: Georgy (RekGRpth) Date: 2016-11-23 04:58
adding to asyncio.Queue class following methods:
    def __aiter__(self): return self
    async def __anext__(self): return await self.get()
let use asyncio.Queue follow:
    queue = asyncio.Queue()
    ...
    async for item in queue: do_something_with(item)
msg281749 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-11-25 23:56
This should be an upstream PR first (GitHub.com/python/asyncio). Also, too late for 3.6.
msg281750 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-11-26 00:29
Also it's not clear that it's a good idea without more thought -- there's no way to end the loop on the producing side.
msg299131 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2017-07-25 18:35
> there's no way to end the loop on the producing side.

I might be missing something, but can't something similar be said of queue.get()?
msg299200 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2017-07-26 03:34
> > there's no way to end the loop on the producing side.

> I might be missing something, but can't something similar be said of queue.get()?

That's my point, actually. If you are wrapping the Queue protocol with __aiter__/__anext__ the caller would expect there's a way to signal to the latter that the loop is over (so it can raise StopAsyncIteration). But since .get() doesn't have a way to signal this, an async for-loop would not be able to terminate (other than through break).
msg299688 - (view) Author: Georgy (RekGRpth) Date: 2017-08-03 07:47
I successfully use my code:

import asyncio, sanic

class MyQueue(asyncio.Queue):
    def __aiter__(self): return self
    async def __anext__(self): return await self.get()

app = sanic.Sanic()
ws_set = set()
app.static('/', 'async.html')

@app.websocket('/ws')
async def root_ws(request, ws):
    ws_set.add(ws)
    try:
        while True: await ws.recv()
    finally: ws_set.remove(ws)

async def postgres():
    import aiopg
    async with aiopg.create_pool('') as pool:
        async with pool.acquire() as connection:
            connection._notifies = MyQueue()
            async with connection.cursor() as cursor:
                await cursor.execute('LISTEN message')
                async for message in connection.notifies:
                    for ws in ws_set: await ws.send(message.payload)

try: asyncio.get_event_loop().run_until_complete(asyncio.gather(app.create_server(), postgres()))
except KeyboardInterrupt: asyncio.get_event_loop().stop()
msg299714 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2017-08-03 16:34
So that's an infinite loop right?
msg300496 - (view) Author: Matt Rasband (Matt Rasband) Date: 2017-08-18 13:32
I attempted this myself, it seemed to have too many costs associated for the stdlib and is something easy enough to wrap myself when I need this functionality with explicit semantics on how to "stop" the queue (using an `object()` sentinel). My implementation is flawed as the sentinel is global, instead of per queue instance, if you decide to use this at all.

See https://github.com/python/asyncio/pull/445
msg300530 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2017-08-18 20:27
Let's not do this.
History
Date User Action Args
2022-04-11 14:58:39adminsetgithub: 72963
2017-08-18 20:27:30gvanrossumsetstatus: open -> closed
resolution: rejected
messages: + msg300530

stage: test needed -> resolved
2017-08-18 13:32:10Matt Rasbandsetnosy: + Matt Rasband
messages: + msg300496
2017-08-03 16:34:27gvanrossumsetmessages: + msg299714
2017-08-03 07:47:41RekGRpthsetmessages: + msg299688
2017-07-26 03:34:51gvanrossumsetmessages: + msg299200
2017-07-25 18:35:07chris.jerdoneksetmessages: + msg299131
2017-07-25 18:21:41chris.jerdoneksetnosy: + chris.jerdonek
2016-11-26 00:29:38gvanrossumsetmessages: + msg281750
2016-11-25 23:56:28gvanrossumsetmessages: + msg281749
versions: - Python 3.5, Python 3.6
2016-11-25 22:26:48terry.reedysettitle: asinc iter queue -> Add asyncio.Queue __aiter__, __anext__ methods
stage: test needed
versions: + Python 3.6, Python 3.7
2016-11-23 04:58:52RekGRpthcreate