Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Prevent double awaiting of async iterator #83567

Closed
jmg mannequin opened this issue Jan 19, 2020 · 8 comments
Closed

Prevent double awaiting of async iterator #83567

jmg mannequin opened this issue Jan 19, 2020 · 8 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes topic-asyncio

Comments

@jmg
Copy link
Mannequin

jmg mannequin commented Jan 19, 2020

BPO 39386
Nosy @njsmith, @asvetlov, @1st1, @miss-islington
PRs
  • bpo-39386: Prevent double awaiting of async iterator #18081
  • [3.8] bpo-39386: Prevent double awaiting of async iterator (GH-18081) #18086
  • [3.7] bpo-39386: Prevent double awaiting of async iterator (GH-18081) #18087
  • bpo-39606: allow closing async generators that are already closed #18475
  • [3.8] bpo-39606: allow closing async generators that are already closed (GH-18475) #18501
  • [3.7] bpo-39606: allow closing async generators that are already closed (GH-18475) #18502
    		•
    Files
  • asyncitertc.py
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-01-20.23:45:40.207>
created_at = <Date 2020-01-19.06:43:40.732>
labels = ['3.7', '3.8', '3.9', 'expert-asyncio']
title = 'Prevent double awaiting of async iterator'
updated_at = <Date 2020-02-13.09:33:42.106>
user = 'https://bugs.python.org/jmg'

    bugs.python.org fields:

    activity = <Date 2020-02-13.09:33:42.106>
actor = 'njs'
assignee = 'none'
closed = True
closed_date = <Date 2020-01-20.23:45:40.207>
closer = 'asvetlov'
components = ['asyncio']
creation = <Date 2020-01-19.06:43:40.732>
creator = 'jmg'
dependencies = []
files = ['48852']
hgrepos = []
issue_num = 39386
keywords = ['patch']
message_count = 8.0
messages = ['360254', '360326', '360340', '360341', '360342', '361942', '361947', '361951']
nosy_count = 5.0
nosy_names = ['jmg', 'njs', 'asvetlov', 'yselivanov', 'miss-islington']
pr_nums = ['18081', '18086', '18087', '18475', '18501', '18502']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue39386'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']
    @jmg
    Copy link
    Mannequin Author

    jmg mannequin commented Jan 19, 2020

    If I create a coro from an async iterator, then wait_for it w/ a timeout, but shielded, so it won't get canceled, and then await upon it, it returns invalid data.

    See the attached test case.

    The reason I do the following is to make sure that an async iterator that I have written doesn't return data early, and needs to wait till later. If I didn't shield it, then the async iterator would get cancelled, and I don't want this.

    I'd expect either correct results to be returned, or an exception to be raised, but in this case, and the docs for wait_for ( https://docs.python.org/3/library/asyncio-task.html#asyncio.wait_for ), I'd expect the correct results to be returned.

    In the attached case, this is the results that I get:
    $python3.7 asyncitertc.py
    3.7.5 (default, Oct 18 2019, 23:59:39)
    [Clang 7.0.2 (clang-700.1.81)]
    timed out
    yielding 1
    results: None
    getting 2: 2

    I do not have python 3.8 to test with.

    @asvetlov
    Copy link
    Contributor

    Thanks for the report.

    Your example could be boiled down to the following:

    coro = asynciter()
await coro
await coro

    The second call incorrectly returns None but should raise an exception, double awaiting is a programming error.

    For regular async functions it raises "RuntimeError: cannot reuse already awaited coroutine"

    @asvetlov asvetlov added 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes labels Jan 20, 2020
    @asvetlov asvetlov changed the title getting invalid data from async iterator Prevent double awaiting of async iterator Jan 20, 2020
    @asvetlov asvetlov added 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes labels Jan 20, 2020
    @asvetlov asvetlov changed the title getting invalid data from async iterator Prevent double awaiting of async iterator Jan 20, 2020
    @asvetlov
    Copy link
    Contributor

    New changeset a96e06d by Andrew Svetlov in branch 'master':
    bpo-39386: Prevent double awaiting of async iterator (GH-18081)
    a96e06d

    @miss-islington
    Copy link
    Contributor

    New changeset 5cadd3f by Miss Islington (bot) in branch '3.8':
    bpo-39386: Prevent double awaiting of async iterator (GH-18081)
    5cadd3f

    @miss-islington
    Copy link
    Contributor

    New changeset b76d5e9 by Miss Islington (bot) in branch '3.7':
    bpo-39386: Prevent double awaiting of async iterator (GH-18081)
    b76d5e9

    @miss-islington
    Copy link
    Contributor

    New changeset 925dc7f by Nathaniel J. Smith in branch 'master':
    bpo-39606: allow closing async generators that are already closed (GH-18475)
    925dc7f

    @miss-islington
    Copy link
    Contributor

    New changeset 8dbdf5f by Miss Islington (bot) in branch '3.8':
    [3.8] bpo-39606: allow closing async generators that are already closed (GH-18475) (GH-18501)
    8dbdf5f

    @njsmith
    Copy link
    Contributor

    njsmith commented Feb 13, 2020

    New changeset f464edf by Nathaniel J. Smith in branch '3.7':
    bpo-39606: allow closing async generators that are already closed (GH-18475) (GH-18502)
    f464edf

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes topic-asyncio
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @asvetlov @njsmith @tirkarthi @miss-islington