Author nmatravolgyi
Recipients Elvis.Pranskevichus, Nikita Ilyasov, aaliddell, asvetlov, chris.jerdonek, lukasz.langa, miss-islington, nmatravolgyi, ods, yselivanov
Date 2021-05-14.19:41:13
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1621021273.6.0.119945375431.issue37658@roundup.psfhosted.org>
In-reply-to
Content
I did a little research on how the `wait_for` could be improved. There are four alternate implementations you can find here: https://github.com/Traktormaster/wait-for2/blob/issue37658/wait_for2/impl.py

The repository (on the linked branch) has tox set-up and a test that asserts the behaviour of builtin and alternate implementations.

Quick summary:
  - wait_for (builtin): either leaks resources or ignores cancellation
  - wait_for (preferred alt): works as expected (I'm going to discuss this below)
  - wait_for_special_raise: works, but has pitfalls
  - wait_for_no_waiter*: These were proposed by @aaliddell on the related PR: https://github.com/python/cpython/pull/26097#issuecomment-840497455 I could not make them work, but I might be missing something. I think the fact that the inner coroutine gets wrapped into a Future introduces the race-condition, but I don't know how to test that. The general idea of this would be the best if an implementation was possible.


About the actually working alternate implementation I made:
In my opinion there is no way to implicitly handle losing a result properly in case of a cancellation, since it arbitrarily breaks the flow of execution. I'd look into having `wait_for(...)` support cleanup callbacks when a cancellation and completion happens at the same time. Something like:

```python

async def create_something():
    # this acquires some resource that needs explicit cleanup after some work
    return object()

def cleanup_something(inst):
    inst.close()

t = asyncio.ensure_future(create_something())
x = await asyncio.wait_for(t, timeout=10, cancel_handler=cleanup_something)
try:
    pass  # normal work with x
finally:
    cleanup_something(x)  # cleanup at normal execution
```

The inner task is still responsible for handling the resource before it returns, which means if the inner task is cancelled, there must be no leak if the implementation is correct. If no cancellation happens, everything is "fine". Finally, the waiter task would be able to handle its cancellation and the simultaneous completion of the inner task if the caller code provides a callback.

Unfortunately this requires the user of `wait_for` to be aware of this race-condition. However it makes it possible to be handled properly when the waited future's result requires a cleanup.
History
Date User Action Args
2021-05-14 19:41:13nmatravolgyisetrecipients: + nmatravolgyi, ods, asvetlov, chris.jerdonek, Elvis.Pranskevichus, lukasz.langa, yselivanov, miss-islington, Nikita Ilyasov, aaliddell
2021-05-14 19:41:13nmatravolgyisetmessageid: <1621021273.6.0.119945375431.issue37658@roundup.psfhosted.org>
2021-05-14 19:41:13nmatravolgyilinkissue37658 messages
2021-05-14 19:41:13nmatravolgyicreate