classification
Title: wait_for(future, ...) should wait for the future (even if a timeout occurs)
Type: Stage:
Components: asyncio Versions: Python 3.8, Python 3.7, Python 3.6, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, giampaolo.rodola, njs, xgdomingo, yselivanov
Priority: normal Keywords:

Created on 2018-02-02 20:01 by njs, last changed 2018-02-11 02:45 by xgdomingo.

Messages (7)
msg311509 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2018-02-02 20:01
Currently, if you use asyncio.wait_for(future, timeout=....) and the timeout expires, then it (a) cancels to the future, and then (b) returns. This is fine if the future is a Future, because Future.cancel is synchronous and completes immediately. But if the future is a Task, then Task.cancel merely requests cancellation, and it will complete later (or not). In particular, this means that wait_for(coro, ...) can return with the coroutine still running, which is surprising.

(Originally encountered by Alex Grönholm, who was using code like

async with aclosing(agen):
    await wait_for(agen.asend(...), timeout=...)

and then confused about why the call to agen.aclose was raising an error complaining that agen.asend was still running. Currently this requires an async_generator based async generator to trigger; with a native async generator, the problem is masked by bpo-32526.)
msg311515 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-02-02 21:02
Looks like a bug.  Andrew, if you have time to look at this, please feel free to go ahead; I'm going to be unavailable till Feb 12 (so I can take a look myself after that).
msg311728 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2018-02-06 15:50
Hmm, I don't see an easy way to fix it.
awaiting for cancelled task potentially can wait forever.
Adding another timeout looks too confusing.
Iterating over a couple of loop steps is not reliable: try/finally block in awaited task can perform async IO with unpredictable completion time.
msg311764 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2018-02-07 06:08
If a task refuses to be cancelled, then is waiting for it forever actually wrong? That's the same thing as happens if I do 'task.cancel(); await task', right? Currently wait_for will abandon such a task, but then it's still left running in the background indefinitely (creating a memory leak or worse).
msg311773 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2018-02-07 08:01
Agree.
Should we report about cancelled but still executing tasks?
It would be a nice feature.
I'm talking not about `wait_for` only but task cancellation in general.
msg311776 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2018-02-07 08:46
How do you tell the difference between a cancelled task that's about to exit, and one that will never exit?
msg311777 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2018-02-07 09:08
Theoretically we can start monitoring cancelled tasks and report about them if the task is still not finished, say, in a minute or two.
It is a new feature, sure.

I'm fine with waiting for cancelled task in wait_for().
History
Date User Action Args
2018-02-11 02:45:45xgdomingosetnosy: + xgdomingo
2018-02-07 09:08:31asvetlovsetmessages: + msg311777
2018-02-07 08:46:38njssetmessages: + msg311776
2018-02-07 08:01:20asvetlovsetmessages: + msg311773
2018-02-07 06:08:36njssetmessages: + msg311764
2018-02-06 15:50:36asvetlovsetmessages: + msg311728
2018-02-02 21:02:20yselivanovsetmessages: + msg311515
2018-02-02 20:01:15njscreate