classification
Title: AsyncIO's wait_for can hide cancellation in a rare race condition
Type: behavior Stage:
Components: asyncio Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, chris.jerdonek, ods, tvoinarovskyi, yselivanov
Priority: normal Keywords:

Created on 2020-10-23 16:10 by tvoinarovskyi, last changed 2020-10-26 06:52 by ods.

Messages (3)
msg379454 - (view) Author: Taras Voinarovskyi (tvoinarovskyi) * Date: 2020-10-23 16:10
Hi, during migration to Python 3.8.6 we encountered a behavior change from previous versions: wait_for ignored the request to cancellation and returned instead. After investigation, it seems to be related to the update in bpo-32751 and is only reproduced if the waited task is finished when cancellation of wait_for happens (code mistakes external CancelledError for a timeout).

The following example can reproduce the behavior on both 3.8.6 and 3.9.0 for me:

```

import asyncio


async def inner():
    return

async def with_for_coro():
    await asyncio.wait_for(inner(), timeout=100)
    await asyncio.sleep(1)
    print('End of with_for_coro. Should not be reached!')

async def main():
    task = asyncio.create_task(with_for_coro())
    await asyncio.sleep(0)
    assert not task.done()
    task.cancel()
    print('Called task.cancel()')
    await task  # -> You would expect a CancelledError to be raised.


asyncio.run(main())
```

Changing the wait time before cancellation slightly will return the correct behavior and CancelledError will be raised.
msg379541 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2020-10-24 21:18
It looks like issue 37658 might be the relevant change rather.

Here is the new logic it introduced: https://github.com/python/cpython/blob/db455296be5f792b8c12b7cd7f3962b52e4f44ee/Lib/asyncio/tasks.py#L483-L488

(via https://github.com/python/cpython/pull/21894 )
msg379577 - (view) Author: Taras Voinarovskyi (tvoinarovskyi) * Date: 2020-10-25 13:07
Hi Chris,
Yes, I do believe that is the respectful change, if we look the CancelledError is not checked to be external or originate from the timer. 
Best regards, 
Taras Voinarovskyi
History
Date User Action Args
2020-10-26 06:52:58odssetnosy: + ods
2020-10-25 13:07:28tvoinarovskyisetmessages: + msg379577
2020-10-24 21:18:28chris.jerdoneksetmessages: + msg379541
2020-10-24 07:53:05chris.jerdoneksetnosy: + chris.jerdonek
2020-10-23 16:10:28tvoinarovskyicreate