classification
Title: Prevent double awaiting of async iterator
Type: Stage: resolved
Components: asyncio Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, jmg, miss-islington, njs, yselivanov
Priority: normal Keywords: patch

Created on 2020-01-19 06:43 by jmg, last changed 2020-02-13 09:33 by njs. This issue is now closed.

Files
File name Uploaded Description Edit
asyncitertc.py jmg, 2020-01-19 06:43
Pull Requests
URL Status Linked Edit
PR 18081 merged asvetlov, 2020-01-20 19:42
PR 18086 merged miss-islington, 2020-01-20 22:49
PR 18087 merged miss-islington, 2020-01-20 22:49
PR 18475 merged njs, 2020-02-12 08:04
PR 18501 merged miss-islington, 2020-02-13 08:19
PR 18502 merged njs, 2020-02-13 08:42
Messages (8)
msg360254 - (view) Author: John-Mark Gurney (jmg) Date: 2020-01-19 06:43
If I create a coro from an async iterator, then wait_for it w/ a timeout, but shielded, so it won't get canceled, and then await upon it, it returns invalid data.

See the attached test case.

The reason I do the following is to make sure that an async iterator that I have written doesn't return data early, and needs to wait till later.  If I didn't shield it, then the async iterator would get cancelled, and I don't want this.

I'd expect either correct results to be returned, or an exception to be raised, but in this case, and the docs for wait_for ( https://docs.python.org/3/library/asyncio-task.html#asyncio.wait_for ), I'd expect the correct results to be returned.


In the attached case, this is the results that I get:
$python3.7 asyncitertc.py 
3.7.5 (default, Oct 18 2019, 23:59:39) 
[Clang 7.0.2 (clang-700.1.81)]
timed out
yielding 1
results: None
getting 2: 2

I do not have python 3.8 to test with.
msg360326 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2020-01-20 18:02
Thanks for the report.

Your example could be boiled down to the following:

coro = asynciter()
await coro
await coro

The second call incorrectly returns None but should raise an exception, double awaiting is a programming error.

For regular async functions it raises "RuntimeError: cannot reuse already awaited coroutine"
msg360340 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2020-01-20 22:49
New changeset a96e06db77dcbd3433d39761ddb4615d7d96284a by Andrew Svetlov in branch 'master':
bpo-39386: Prevent double awaiting of async iterator (GH-18081)
https://github.com/python/cpython/commit/a96e06db77dcbd3433d39761ddb4615d7d96284a
msg360341 - (view) Author: miss-islington (miss-islington) Date: 2020-01-20 23:06
New changeset 5cadd3fe3aead1b5bee1438dc03383d6739d4209 by Miss Islington (bot) in branch '3.8':
bpo-39386: Prevent double awaiting of async iterator (GH-18081)
https://github.com/python/cpython/commit/5cadd3fe3aead1b5bee1438dc03383d6739d4209
msg360342 - (view) Author: miss-islington (miss-islington) Date: 2020-01-20 23:07
New changeset b76d5e9ee64b8ac861e856d8e41289c0977c75e1 by Miss Islington (bot) in branch '3.7':
bpo-39386: Prevent double awaiting of async iterator (GH-18081)
https://github.com/python/cpython/commit/b76d5e9ee64b8ac861e856d8e41289c0977c75e1
msg361942 - (view) Author: miss-islington (miss-islington) Date: 2020-02-13 08:15
New changeset 925dc7fb1d0db85dc137afa4cd14211bf0d67414 by Nathaniel J. Smith in branch 'master':
bpo-39606: allow closing async generators that are already closed (GH-18475)
https://github.com/python/cpython/commit/925dc7fb1d0db85dc137afa4cd14211bf0d67414
msg361947 - (view) Author: miss-islington (miss-islington) Date: 2020-02-13 08:43
New changeset 8dbdf5f275c6462bb522bcf3a29054239d72989d by Miss Islington (bot) in branch '3.8':
[3.8] bpo-39606: allow closing async generators that are already closed (GH-18475) (GH-18501)
https://github.com/python/cpython/commit/8dbdf5f275c6462bb522bcf3a29054239d72989d
msg361951 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2020-02-13 09:33
New changeset f464edf3239f7867fe31c9cd238a68fb3b90feaa by Nathaniel J. Smith in branch '3.7':
bpo-39606: allow closing async generators that are already closed (GH-18475) (GH-18502)
https://github.com/python/cpython/commit/f464edf3239f7867fe31c9cd238a68fb3b90feaa
History
Date User Action Args
2020-02-13 09:33:42njssetnosy: + njs
messages: + msg361951
2020-02-13 08:43:28miss-islingtonsetmessages: + msg361947
2020-02-13 08:42:09njssetpull_requests: + pull_request17878
2020-02-13 08:19:50miss-islingtonsetpull_requests: + pull_request17876
2020-02-13 08:15:45miss-islingtonsetmessages: + msg361942
2020-02-12 08:04:59njssetpull_requests: + pull_request17844
2020-01-20 23:45:40asvetlovsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-01-20 23:07:58miss-islingtonsetmessages: + msg360342
2020-01-20 23:06:44miss-islingtonsetnosy: + miss-islington
messages: + msg360341
2020-01-20 22:49:53miss-islingtonsetpull_requests: + pull_request17479
2020-01-20 22:49:46miss-islingtonsetpull_requests: + pull_request17478
2020-01-20 22:49:37asvetlovsetmessages: + msg360340
2020-01-20 19:42:05asvetlovsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request17473
2020-01-20 18:03:03asvetlovsettitle: getting invalid data from async iterator -> Prevent double awaiting of async iterator
versions: + Python 3.7, Python 3.8, Python 3.9
2020-01-20 18:02:13asvetlovsetmessages: + msg360326
stage: needs patch
2020-01-19 07:18:19xtreaksetnosy: + asvetlov, yselivanov
components: + asyncio
2020-01-19 06:43:40jmgcreate