Here is the minimal example code:

https://gist.github.com/achimnol/965a6aecf7b1f96207abf11469b68965

Just run this code using "python -m pytest -s test.py" to see what happens.
(My setup uses Python 3.6.4 and pytest 3.3.2 on macOS High Sierra 10.13.2)

I tried the same logic using synchornous APIs such as threading.Thread / time.sleep instead of loop.create_task / asyncio.sleep, it raised "ValueError: generator already executing" when tried to close the generator.

It looks like Python's tracking the "running" state of async generators wrong: we should have ag_running set to True when we enter asend/athrow/aclose and False when we exit, but instead it's being toggled back and forth on each *inner* send/throw on the individual coroutines.

Here's a minimal reproducer (using some random recent checkout of master):

>>> async def f():
...     await asyncio.sleep(1)
...     yield
... 
>>> ag = f()
>>> asend_coro = ag.asend(None)
>>> fut = asend_coro.send(None)
# Logically, ag.asend is still running, waiting for that sleep to
# finish, but we have lost track:
>>> ag.ag_running
False
# We can start another call to asend() going simultaneously
>>> fut.set_result(None)
>>> send_coro2 = ag.asend(None)
>>> send_coro2.send(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

It looks like async_generator did handle this case correctly, but didn't have a test case. I just added one: https://github.com/njsmith/async_generator/commit/339fc6309aa6c96244e79b517db0b98ba0ccfb2a

Was fixed as part of #30773.