classification
Title: Async generator might re-throw GeneratorExit on aclose()
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Roman.Evstifeev, catern, miss-islington, vxgmichel
Priority: normal Keywords: patch

Created on 2018-12-04 13:21 by vxgmichel, last changed 2020-08-29 20:34 by catern. This issue is now closed.

Files
File name Uploaded Description Edit
example.py vxgmichel, 2018-12-04 13:21 Reproduce the issue with asyncio
test.py vxgmichel, 2018-12-04 13:22 Reproduce the issue without asyncio
patch.diff vxgmichel, 2018-12-04 13:22 A possible fix
Pull Requests
URL Status Linked Edit
PR 14755 merged vxgmichel, 2019-07-13 16:04
PR 17257 merged miss-islington, 2019-11-19 13:54
PR 17258 merged miss-islington, 2019-11-19 13:54
Messages (6)
msg331043 - (view) Author: Vincent Michel (vxgmichel) * Date: 2018-12-04 13:21
As far as I can tell, this issue is different than: https://bugs.python.org/issue34730

I noticed `async_gen.aclose()` raises a GeneratorExit exception if the async generator finalization awaits and silence a failing unfinished future (see example.py).

This seems to be related to a bug in `async_gen_athrow_throw`. In fact, `async_gen.aclose().throw(exc)` does not silence GeneratorExit exceptions. This behavior can be reproduced without asyncio (see test.py).

Attached is a possible patch, although I'm not too comfortable messing with the python C internals. I can make a PR if necessary.
msg356968 - (view) Author: miss-islington (miss-islington) Date: 2019-11-19 13:53
New changeset 8e0de2a4808d7c2f4adedabff89ee64e0338790a by Miss Islington (bot) (Vincent Michel) in branch 'master':
bpo-35409: Ignore GeneratorExit in async_gen_athrow_throw (GH-14755)
https://github.com/python/cpython/commit/8e0de2a4808d7c2f4adedabff89ee64e0338790a
msg356969 - (view) Author: miss-islington (miss-islington) Date: 2019-11-19 14:12
New changeset 6c3b471c8c0bfd49c664d8ee7e95da3710fd6069 by Miss Islington (bot) in branch '3.8':
bpo-35409: Ignore GeneratorExit in async_gen_athrow_throw (GH-14755)
https://github.com/python/cpython/commit/6c3b471c8c0bfd49c664d8ee7e95da3710fd6069
msg356970 - (view) Author: miss-islington (miss-islington) Date: 2019-11-19 14:12
New changeset 4ffc569b47bef9f95e443f3c56f7e7e32cb440c0 by Miss Islington (bot) in branch '3.7':
bpo-35409: Ignore GeneratorExit in async_gen_athrow_throw (GH-14755)
https://github.com/python/cpython/commit/4ffc569b47bef9f95e443f3c56f7e7e32cb440c0
msg376075 - (view) Author: Spencer Baugh (catern) Date: 2020-08-29 20:32
I'm not sure this was the correct fix - or at least, this creates further issues with asynccontextmanager. Consider the following code:

---------------
import contextlib
import types

@contextlib.asynccontextmanager
async def acm():
    # GeneratorExit athrown here from AsyncContextManager __aexit__,
    # propagated from the body of the contextmanager in func()
    yield

@types.coroutine
def _yield():
    yield

async def func():
    async with acm():
        # GeneratorExit raised here
        await _yield()

x = func()
x.send(None) # start running func
x.close() # raise GeneratorExit in func at its current yield
# AsyncContextManager __aexit__ fails with "RuntimeError: generator didn't stop after throw()"
---------------

The reason for the failure in AsyncContextManager __aexit__ is that the asyncgenerator raises StopIteration instead of GeneratorExit when agen.athrow(GeneratorExit()) is called and driven, so "await agen.athrow(GeneratorExit())" just evaluates to None, rather than raising GeneratorExit.

On 3.6 this would work fine, because "await athrow(GeneratorExit())" will raise GeneratorExit. I suspect this was broken by this change.
msg376076 - (view) Author: Spencer Baugh (catern) Date: 2020-08-29 20:34
My mistake, I see now this is just https://bugs.python.org/issue33786 and is already fixed.
History
Date User Action Args
2020-08-29 20:34:14caternsetmessages: + msg376076
2020-08-29 20:32:23caternsetnosy: + catern
messages: + msg376075
2020-05-24 06:22:47Roman.Evstifeevsetnosy: + Roman.Evstifeev
2019-11-19 14:12:24miss-islingtonsetmessages: + msg356970
2019-11-19 14:12:17miss-islingtonsetmessages: + msg356969
2019-11-19 13:55:48asvetlovsetstatus: open -> closed
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.9, - Python 3.6
2019-11-19 13:54:47miss-islingtonsetpull_requests: + pull_request16752
2019-11-19 13:54:10miss-islingtonsetpull_requests: + pull_request16751
2019-11-19 13:53:56miss-islingtonsetnosy: + miss-islington
messages: + msg356968
2019-07-13 16:04:04vxgmichelsetstage: patch review
pull_requests: + pull_request14550
2018-12-04 13:22:28vxgmichelsetfiles: + patch.diff
keywords: + patch
2018-12-04 13:22:10vxgmichelsetfiles: + test.py
2018-12-04 13:21:20vxgmichelcreate