classification
Title: Regression: it should be possible to close async iterators multiple times
Type: Stage: resolved
Components: Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, lukasz.langa, miss-islington, ned.deily, njs, yselivanov
Priority: release blocker Keywords: 3.7regression, 3.8regression, 3.9regression, patch

Created on 2020-02-11 06:35 by njs, last changed 2020-02-13 09:44 by njs. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 18475 merged njs, 2020-02-12 08:04
PR 18501 merged miss-islington, 2020-02-13 08:19
PR 18502 merged njs, 2020-02-13 08:42
Messages (4)
msg361783 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2020-02-11 06:35
In bpo-39386, the 'aclose' method for async generators was fixed so that the following broken code would correctly raise an error:

# -- bad code --
async def agen_fn():
    yield

async def do_bad_thing():
    agen = agen_fn()
    aclose_coro = agen.aclose()
    await aclose_coro
    # Should raise RuntimeError:
    await aclose_coro

asyncio.run(do_bad_thing())

# -----------------

However, the merged fix also broke the following correct code, which should *not* raise an error:

# -- good code --
async def agen_fn():
    yield

async def do_good_thing():
    agen = agen_fn()
    await agen.aclose()
    # Should *not* raise an error, but currently does in Python dev branches:
    await agen.aclose()

asyncio.run(do_good_thing())

# ----------------

It's not supported to iterate the same coroutine object twice. However, making two *independent* calls to aclose should return two independent coroutine objects, and it should be legal to iterate over each object once.

This can also occur even if there's only a single call to 'aclose'. For example, this is the recommended idiom for making sure that an async generator correctly closes all its resources:

# -- good code --
async def agen_fn():
    yield

async def careful_loop():
    agen = agen_fn()
    try:
        async for _ in agen:
            pass
    finally:
        await agen.aclose()

asyncio.run(careful_loop())

# -------------------

On released Python, the code above runs correctly. On the latest Python dev branches, it raises a RuntimeError.

So basically the problem is that the fix for bpo-39386 is storing the "was aclose iterated?" state on the async generator object, but in fact it needs to be stored on the aclose coroutine object.
msg361943 - (view) Author: miss-islington (miss-islington) Date: 2020-02-13 08:15
New changeset 925dc7fb1d0db85dc137afa4cd14211bf0d67414 by Nathaniel J. Smith in branch 'master':
bpo-39606: allow closing async generators that are already closed (GH-18475)
https://github.com/python/cpython/commit/925dc7fb1d0db85dc137afa4cd14211bf0d67414
msg361948 - (view) Author: miss-islington (miss-islington) Date: 2020-02-13 08:43
New changeset 8dbdf5f275c6462bb522bcf3a29054239d72989d by Miss Islington (bot) in branch '3.8':
[3.8] bpo-39606: allow closing async generators that are already closed (GH-18475) (GH-18501)
https://github.com/python/cpython/commit/8dbdf5f275c6462bb522bcf3a29054239d72989d
msg361952 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2020-02-13 09:33
New changeset f464edf3239f7867fe31c9cd238a68fb3b90feaa by Nathaniel J. Smith in branch '3.7':
bpo-39606: allow closing async generators that are already closed (GH-18475) (GH-18502)
https://github.com/python/cpython/commit/f464edf3239f7867fe31c9cd238a68fb3b90feaa
History
Date User Action Args
2020-02-13 09:44:45njssetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-02-13 09:33:42njssetmessages: + msg361952
2020-02-13 08:43:28miss-islingtonsetmessages: + msg361948
2020-02-13 08:42:09njssetpull_requests: + pull_request17877
2020-02-13 08:19:49miss-islingtonsetpull_requests: + pull_request17875
2020-02-13 08:15:45miss-islingtonsetnosy: + miss-islington
messages: + msg361943
2020-02-12 08:04:59njssetkeywords: + patch
stage: patch review
pull_requests: + pull_request17843
2020-02-11 06:35:23njscreate