classification
Title: async generator receives wrong value when shared between coroutines
Type: behavior Stage: patch review
Components: Interpreter Core Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: yselivanov Nosy List: Dima.Tisnek, chris.jerdonek, emilyemorehouse, jan.cespivo, njs, xtreak, yselivanov
Priority: high Keywords: patch

Created on 2017-06-26 18:55 by Dima.Tisnek, last changed 2018-09-22 16:47 by xtreak.

Pull Requests
URL Status Linked Edit
PR 7468 open yselivanov, 2018-06-07 05:02
Messages (7)
msg296931 - (view) Author: Dima Tisnek (Dima.Tisnek) * Date: 2017-06-26 18:55
MRE

```
import asyncio


async def generator():
    while True:
        x = yield 42
        print("received", x)
        await asyncio.sleep(0.1)


async def user(name, g):
    print("sending", name)
    await g.asend(name)


async def helper():
    g = generator()
    await g.asend(None)

    await asyncio.gather(*(user(f"user-{x}", g) for x in range(3)))


if __name__ == "__main__":
    asyncio.get_event_loop().run_until_complete(helper())
```

Produces output:

```
sending user-0
received user-0
sending user-1
sending user-2
received None
received None
```

Expected output (some variance allowed):

```
sending user-0
received user-0
sending user-1
sending user-2
received user-1
received user-2
```

Initial report / discussion: https://mail.python.org/pipermail/async-sig/2017-June/000293.html
msg296932 - (view) Author: Dima Tisnek (Dima.Tisnek) * Date: 2017-06-26 19:03
@Yuri, this bug doesn't require `gather`, here's a version with futures and explicit await's instead. It produces same output:

```
import asyncio


async def generator():
    while True:
        x = yield 42
        print("received", x)
        await asyncio.sleep(0.1)


async def user(name, g):
    print("sending", name)
    await g.asend(name)


async def helper():
    g = generator()
    await g.asend(None)

    u0 = asyncio.ensure_future(user("user-0", g))
    u1 = asyncio.ensure_future(user("user-1", g))
    u2 = asyncio.ensure_future(user("user-2", g))

    await u0
    await u1
    await u2


if __name__ == "__main__":
    asyncio.get_event_loop().run_until_complete(helper())
```

Same with `asyncio.get_event_loop().create_task` as well.
msg297022 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2017-06-27 11:27
Note that the example can be further simplified by replacing user() with:

    async def send_hello(g):
        print("sending: hello")
        await g.asend("hello")

Then the output is:

sending: hello
received hello
sending: hello
sending: hello
received None
received None
msg317563 - (view) Author: Jan Češpivo (jan.cespivo) Date: 2018-05-24 11:49
I've reproduced the problem also in 3.7 branch.

```
import asyncio

loop = asyncio.get_event_loop()


async def consumer():
    while True:
        await asyncio.sleep(0)
        message = yield
        print('received', message)


async def amain():
    agenerator = consumer()
    await agenerator.asend(None)

    fa = asyncio.create_task(agenerator.asend('A'))
    fb = asyncio.create_task(agenerator.asend('B'))
    await fa
    await fb


loop.run_until_complete(amain())
```

Output:
```
received A
received None
```

If the line `await asyncio.sleep(0)` is omitted the output is ok:
```
received A
received B
```
msg317603 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-05-24 18:39
Thanks Jan. Thanks a lot for a short script to reproduce this bug.

The actual problem here is that asynchronous generators don't control their 'asend' and 'athrow' coroutines in any way. So if you have two of them iterating *in parallel* they will cause their asynchronous generator to be in an inconsistent state.

The most obvious solution to this problem is to prohibit iterating 'asend'/'athrow' objects in parallel by throwing an exception.  

Nathaniel, what are your thoughts on this?
msg317605 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2018-05-24 19:14
My thoughts: https://bugs.python.org/issue32526#msg309783
msg317607 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-05-24 19:19
Thanks, I'll look into adding ag_running properly.
History
Date User Action Args
2018-09-22 16:47:30xtreaksetnosy: + xtreak
2018-06-07 05:02:30yselivanovsetkeywords: + patch
stage: patch review
pull_requests: + pull_request7091
2018-05-24 19:19:56yselivanovsetmessages: + msg317607
2018-05-24 19:14:14njssetmessages: + msg317605
2018-05-24 18:39:39yselivanovsetpriority: normal -> high
assignee: yselivanov
components: + Interpreter Core, - asyncio
versions: + Python 3.8
2018-05-24 18:39:04yselivanovsetnosy: + njs
messages: + msg317603
2018-05-24 11:49:24jan.cespivosetnosy: + jan.cespivo

messages: + msg317563
versions: + Python 3.7
2017-06-27 11:27:00chris.jerdoneksetnosy: + chris.jerdonek
messages: + msg297022
2017-06-27 01:33:10emilyemorehousesetnosy: + emilyemorehouse
2017-06-26 19:03:36Dima.Tisneksetmessages: + msg296932
2017-06-26 18:55:33Dima.Tisnekcreate