classification
Title: Odd error awaiting a Future
Type: Stage:
Components: asyncio Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Dima.Tisnek, asvetlov, yselivanov
Priority: normal Keywords:

Created on 2019-06-06 10:29 by Dima.Tisnek, last changed 2019-06-10 10:18 by Dima.Tisnek.

Messages (9)
msg344798 - (view) Author: Dima Tisnek (Dima.Tisnek) * Date: 2019-06-06 10:29
Let's start with correct code:


import asyncio


async def writer():
    await asyncio.sleep(1)
    g1.set_result(41)

async def reader():
    await g1

async def test():
    global g1
    g1 = asyncio.Future()
    await asyncio.gather(reader(), writer())

asyncio.run(test())


No error, as expected.

Now let's mess it up a bit:


import asyncio

g1 = asyncio.Future()


async def writer():
    await asyncio.sleep(1)
    g1.set_result(41)

async def reader():
    await g1

async def test():
    await asyncio.gather(reader(), writer())

asyncio.run(test())


Fails with RuntimeError ... attached to a different loop

The error makes sense, although it's sad that I can't create global futures / there was no even loop when Future was creates, it was not a *different* event loop / maybe I wish .run() didn't force a new event loop?

A nit (IMO), but I can live with it.

Let's mess the code up a bit more:


import asyncio

g1 = asyncio.Future()


async def writer():
    await asyncio.sleep(1)
    g1.set_result(41)

async def reader():
    await g1

async def test():
    await asyncio.gather(reader(), reader(), writer())

asyncio.run(test())


RuntimeError: await wasn't used with future

What?
That's really confusing!
The only difference is that there are now 2 readers running in parallel.

The actual exception comes from asyncio.Future.__await__ after a yield.
I'm not sure how to fix this...
msg344800 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-06-06 11:10
Global future objects (and global asyncio objects in general) don't work with asyncio.run(). The lifecycle of these objects should be closer than loop but asyncio.run() creates a loop during execution.

I think this is a good design, we have a plan to deprecate and eventually drop all old-times mess that allows confusions.

All three of your problems are because you use a global future which is implicitly attached to the different loop.

Also I'd like to note that futures are low-level API, which is very delicate and error-prone. Futures are crucial for libraries building (e.g. aiohttp uses them a lot) but working with futures in application code is an explicit sign of bad design.
msg345017 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-06-08 01:40
Dima, unless you want to make a specific doc change suggestion, I think this should be closed.  The planned code changes will be on other issues.
msg345100 - (view) Author: Dima Tisnek (Dima.Tisnek) * Date: 2019-06-10 01:48
Hi Terry,

Yes, I have a specific suggestion:

The error `RuntimeError: await wasn't used with future` is misleading.
I'm not sure if changing error text is enough.
I think that Future.__await__ should be fixed; I think that `await f` should be idempotent, that the same exception should be raised in single and double await examples.
msg345102 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-06-10 01:54
OK.  I am quitting here because asyncio and futures are outside my expertise.
msg345111 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-06-10 09:37
Not sure what "idempotency fix" means in the context of Future objects.

Could you describe desired behavior or, even better, provide a pull request that demonstrates your desire?
msg345114 - (view) Author: Dima Tisnek (Dima.Tisnek) * Date: 2019-06-10 09:57
I think that if a Future is abused, the following two should return *same* error:

async def test():
    await asyncio.gather(reader(), writer())

-vs-

async def test():
    await asyncio.gather(reader(), reader(), writer())
msg345115 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-06-10 10:05
Agree, it would be fine.
So the behavior is not changed but the error text should be clear, right?
msg345118 - (view) Author: Dima Tisnek (Dima.Tisnek) * Date: 2019-06-10 10:18
Yes, I think that would be sufficient!

I don't know about implementation though, as there's some magic in the way Future.__await__ tries to guess how it was called. I hope one day that magic could go away, but I neither understand the origin nor the mechanics of it, so... My gut tells me that fixing the error involves digging into that magic.
History
Date User Action Args
2019-06-10 10:18:40Dima.Tisneksetmessages: + msg345118
2019-06-10 10:05:04asvetlovsetmessages: + msg345115
2019-06-10 09:57:54Dima.Tisneksetmessages: + msg345114
2019-06-10 09:37:25asvetlovsetmessages: + msg345111
2019-06-10 01:54:54terry.reedysetnosy: - terry.reedy
2019-06-10 01:54:44terry.reedysetmessages: + msg345102
2019-06-10 01:48:56Dima.Tisneksetmessages: + msg345100
2019-06-08 01:40:13terry.reedysetnosy: + terry.reedy
messages: + msg345017
2019-06-07 23:07:59terry.reedysettitle: Odd error awating a Future -> Odd error awaiting a Future
versions: + Python 3.9
2019-06-06 11:10:52asvetlovsetmessages: + msg344800
2019-06-06 10:29:59Dima.Tisnekcreate