This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Worse error from asynccontextmanager in Python 3.10
Type: behavior Stage: resolved
Components: asyncio Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: Dima.Tisnek, asvetlov, graingert, lukasz.langa, ncoghlan, yselivanov
Priority: normal Keywords:

Created on 2021-12-06 13:24 by Dima.Tisnek, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (10)
msg407799 - (view) Author: Dima Tisnek (Dima.Tisnek) * Date: 2021-12-06 13:24
Consider this illegal code:



import logging
from asyncio import sleep, gather, run
from contextlib import asynccontextmanager

@asynccontextmanager
async def foo():
    await sleep(1)
    yield


async def test():
    f = foo()
    await gather(f.__aenter__(), f.__aenter__())

run(test())



If it's ran with Python 3.9, user gets a sensible error:


  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py", line 175, in __aenter__
    return await self.gen.__anext__()
RuntimeError: anext(): asynchronous generator is already running



However, if it's ran with Python 3.10, user gets a cryptic error:


  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py", line 197, in __aenter__
    del self.args, self.kwds, self.func
AttributeError: args



Which makes it harder to pinpoint what's wrong when the stack is complex.
I've hit this with fastapi/starlette/mangum combo and a custom middleware.
msg407811 - (view) Author: Thomas Grainger (graingert) * Date: 2021-12-06 14:18
I think `AttributeError: args` is the desired/expected behaviour

consider the sync version:

```
import logging
from asyncio import sleep, gather, run
from contextlib import asynccontextmanager, contextmanager

@contextmanager
def foo():
    yield


def test():
    f = foo()
    f.__enter__()
    f.__enter__()

test()
```

```
Traceback (most recent call last):
  File "/home/graingert/projects/example/sync.py", line 15, in <module>
    test()
  File "/home/graingert/projects/example/sync.py", line 13, in test
    f.__enter__()
  File "/usr/lib/python3.9/contextlib.py", line 117, in __enter__
    del self.args, self.kwds, self.func
AttributeError: args
```
msg407812 - (view) Author: Thomas Grainger (graingert) * Date: 2021-12-06 14:19
or consider the trio version:

```
import logging
import trio
from contextlib import asynccontextmanager

@asynccontextmanager
async def foo():
    await trio.sleep(1)
    yield


async def test():
    async with trio.open_nursery() as n:
        f = foo()
        n.start_soon(f.__aenter__)
        n.start_soon(f.__aenter__)

trio.run(test)
```

```
Traceback (most recent call last):
  File "/home/graingert/projects/examples/bar.py", line 17, in <module>
    trio.run(test)
  File "/home/graingert/.virtualenvs/testing39/lib/python3.9/site-packages/trio/_core/_run.py", line 1932, in run
    raise runner.main_task_outcome.error
  File "/home/graingert/projects/examples/bar.py", line 15, in test
    n.start_soon(f.__aenter__)
  File "/home/graingert/.virtualenvs/testing39/lib/python3.9/site-packages/trio/_core/_run.py", line 815, in __aexit__
    raise combined_error_from_nursery
  File "/usr/lib/python3.9/contextlib.py", line 179, in __aenter__
    del self.args, self.kwds, self.func
AttributeError: args
```
msg407813 - (view) Author: Thomas Grainger (graingert) * Date: 2021-12-06 14:25
ah I can repeat this on python3.8.10 trio but not python3.9.9 trio:

```
import logging
import trio
from contextlib import asynccontextmanager

@asynccontextmanager
async def foo():
    await trio.sleep(1)
    yield


async def test():
    async with trio.open_nursery() as n:
        f = foo()
        n.start_soon(f.__aenter__)
        n.start_soon(f.__aenter__)

trio.run(test)

```

```
Traceback (most recent call last):
  File "bar.py", line 17, in <module>
    trio.run(test)
  File "/home/graingert/.virtualenvs/osirium-main/lib/python3.8/site-packages/trio/_core/_run.py", line 1932, in run
    raise runner.main_task_outcome.error
  File "bar.py", line 15, in test
    n.start_soon(f.__aenter__)
  File "/home/graingert/.virtualenvs/osirium-main/lib/python3.8/site-packages/trio/_core/_run.py", line 815, in __aexit__
    raise combined_error_from_nursery
  File "/usr/lib/python3.8/contextlib.py", line 171, in __aenter__
    return await self.gen.__anext__()
RuntimeError: anext(): asynchronous generator is already running
```
msg407814 - (view) Author: Thomas Grainger (graingert) * Date: 2021-12-06 14:30
I see the change here: https://github.com/python/cpython/commit/1c5c9c89ffc36875afaf4c3cc6a716d4bd089bbf#diff-e00601a380ba6c916ba4333277fe6ea43d2477804002ab1ae64480f80fec8e3aR177-R179

this is intentionally implementing https://bugs.python.org/issue30306 for asynccontextmanagers that was initially missing
msg407817 - (view) Author: Thomas Grainger (graingert) * Date: 2021-12-06 14:34
you can see the analogous sync contextmanager issue on python3.6 with:

```
import logging
from contextlib import contextmanager

@contextmanager
def foo():
    yield


def test():
    f = foo()
    f.__enter__()
    f.__enter__()

test()
```

on python3.7+ you get the bpo-30306 behaviour

```
Traceback (most recent call last):
  File "sync.py", line 14, in <module>
    test()
  File "sync.py", line 12, in test
    f.__enter__()
  File "/usr/lib/python3.8/contextlib.py", line 111, in __enter__
    del self.args, self.kwds, self.func
AttributeError: args
```

and python3.6 you get the same sort of error you see now for asynccontextmanagers:

```
Traceback (most recent call last):
  File "sync.py", line 14, in <module>
    test()
  File "sync.py", line 12, in test
    f.__enter__()
  File "/usr/lib/python3.6/contextlib.py", line 83, in __enter__
    raise RuntimeError("generator didn't yield") from None
RuntimeError: generator didn't yield
```
msg408939 - (view) Author: Dima Tisnek (Dima.Tisnek) * Date: 2021-12-20 01:55
Andrew, I see that you've closed this issue as "fixed".

I'm a little confused by that.

If you mean that 3.10 behaviour is better than 3.9, than perhaps "not a bug / rejected / wont fix" would make more sense.

Actually I don't agree with Thomas's logic... his argument feels like consistency for its own sake.

I agree with the intention of the change for the arguments to be released (https://bugs.python.org/issue30306) but I disagree with the cryptic error.

I'm sure we could have both argument release and readable exception, e.g.:


try:
    del self.args, ...
except AttributeError:
    raise RuntimeError("anext(): asynchronous generator is already running")
msg408940 - (view) Author: Thomas Grainger (graingert) * Date: 2021-12-20 02:04
> Actually I don't agree with Thomas's logic... his argument feels like consistency for its own sake.

Do you expect sync and async contextmanagers to act differently?

Why would sync contextmanagers raise AttributeError and async contextmanagers raise a RuntimeError?

If it's sensible to guard against invalid re-entry for async contextmanagers then I think it's sensible to apply the same guard to sync contextmanagers.
msg408941 - (view) Author: Dima Tisnek (Dima.Tisnek) * Date: 2021-12-20 02:15
I'm fine with guarding both.
msg408948 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2021-12-20 08:04
Sorry, I closed it because async behavior reflects sync version now.

If you want to improve both -- you are welcome! Perhaps it is worth another issue with another problem description.
History
Date User Action Args
2022-04-11 14:59:53adminsetgithub: 90154
2021-12-20 08:04:18asvetlovsetresolution: fixed -> wont fix
messages: + msg408948
2021-12-20 02:15:19Dima.Tisneksetmessages: + msg408941
2021-12-20 02:04:04graingertsetmessages: + msg408940
2021-12-20 01:55:03Dima.Tisneksetmessages: + msg408939
2021-12-19 17:16:00asvetlovsetstatus: open -> closed
resolution: fixed
stage: resolved
2021-12-06 14:34:48graingertsetmessages: + msg407817
2021-12-06 14:30:12graingertsetmessages: + msg407814
2021-12-06 14:25:35graingertsetmessages: + msg407813
2021-12-06 14:19:07graingertsetmessages: + msg407812
2021-12-06 14:18:22graingertsetmessages: + msg407811
2021-12-06 13:49:24serhiy.storchakasetnosy: + ncoghlan, lukasz.langa, graingert
2021-12-06 13:24:19Dima.Tisnekcreate