classification
Title: error message when __aexit__ is not async
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: yselivanov Nosy List: Chewy, Mark.Shannon, RazerM, Tadhg McDonald-Jensen, asvetlov, levkivskyi, miss-islington, ned.deily, r.david.murray, serhiy.storchaka, yselivanov
Priority: normal Keywords: patch

Created on 2017-03-27 17:24 by Tadhg McDonald-Jensen, last changed 2018-04-04 17:39 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 6352 merged serhiy.storchaka, 2018-04-02 21:36
PR 6353 merged miss-islington, 2018-04-02 22:42
PR 6354 merged miss-islington, 2018-04-02 22:42
PR 6370 merged serhiy.storchaka, 2018-04-04 14:08
PR 6371 merged miss-islington, 2018-04-04 15:46
PR 6373 merged miss-islington, 2018-04-04 16:30
Messages (19)
msg290630 - (view) Author: Tadhg McDonald-Jensen (Tadhg McDonald-Jensen) * Date: 2017-03-27 17:24
When creating a asynchronous context manager if the __aexit__ method is not labeled as async (so it returns None instead of a coroutine) the error has a generic error message:

TypeError: object NoneType can't be used in 'await' expression

Would it be possible to change this so it indicates that it was the context that was invalid not an `await` statement?  Since the traceback points to the last statement of the with block it can create very confusing errors if the last statement was an await.

Example:

import asyncio
class Test():
    async def __aenter__(self):
        print("aenter used")
        value = asyncio.Future()
        value.set_result(True)
        return value
    #FORGOT TO MARK AS async !!
    def __aexit__(self, *errors):
        print("aexit used")
        return None

async def my_test():
    async with Test() as x:
        print("inside async with, now awaiting on", x)
        await x

my_test().send(None)

Give the output:

aenter used
inside async with, now awaiting on <Future finished result=True>
aexit used
Traceback (most recent call last):
  File ".../test.py", line 19, in <module>
    my_test().send(None)
  File ".../test.py", line 16, in my_test
    await x
TypeError: object NoneType can't be used in 'await' expression

Which indicates to me that `x` was None when it was await-ed for.
msg290631 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-03-27 17:33
This is a specific example of the general problem reported in issue 25538.
msg290633 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2017-03-27 17:46
> This is a specific example of the general problem reported in issue 25538.

It's a bit different code path/problem.  But I agree, we should work on making both `with` and `async with` a bit more usable.

> Would it be possible to change this so it indicates that it was the context that was invalid not an `await` statement?

I'm not sure if we can do anything about this.  We compile 'async with' into a set of opcodes.  The first ones resolve __aexit__, the latter ones await on it. The one that prepares to await on the aexit (GET_AWAITABLE) is the same that 'await' expression compiles to, and that opcode has no idea what exactly it awaits on.

We could probably add GET_AEXIT_AWAITABLE opcode specifically to improve the error message (other than that it would be a copy of GET_AWAITABLE). I'll think about it.
msg290639 - (view) Author: Tadhg McDonald-Jensen (Tadhg McDonald-Jensen) * Date: 2017-03-27 19:53
> This is a specific example of the general problem reported in issue 25538.

Definitely related, although the part of giving an error "... can't be used in 'await' expression" when no await expression is used isn't covered by that thread so I'm not sure I'd call this a direct duplicate.


Currently when __anext__ return a non-awaitable this error is shown:

TypeError: 'async for' received an invalid object from __anext__: NoneType

So having __aenter__ and __aexit__ have similar error messages would be nice.

Would it maybe make sense to implement this as adding an argument to GET_AWAITABLE to indicate whether it was called from await, __anext__, __aenter__ or __aexit__? (or others that may exist in future)  I don't know enough about bytecode to know how it'd compare to an instruction for each case.
msg290640 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2017-03-27 19:56
> Would it maybe make sense to implement this as adding an argument to GET_AWAITABLE to indicate whether it was called from await, __anext__, __aenter__ or __aexit__?

Yes, but it will make it a tad slower (which will also affect await performance).  I'll run some benchmarks.
msg308797 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2017-12-20 20:02
It's Interpreter Core problem, not specific to asyncio bug.
msg314827 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-04-02 21:48
This issue looks unrelated to issue25538.

PR 6352 improves error messages without changing bytecode. It is so simple that could be backported to 3.7 if Ned approves this.

But it conflicts with issue32949. This way will not work after merging any of current issue32949 PRs. We will have to find other way for issue32949.
msg314831 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-04-02 22:23
PR 6352 smells like a bug fix to me and I think it should be OK for 3.7.0b4.
msg314832 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-04-02 22:28
> PR 6352 smells like a bug fix to me and I think it should be OK for 3.7.0b4.

Can we backport it to 3.6 too?
msg314834 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-04-02 22:31
> Can we backport it to 3.6 too?

I suppose so.  (Though you'll owe me if you break anything.)
msg314836 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-04-02 22:41
New changeset a68f2f0578bbf812fa2264d0e0bb388340d6e230 by Serhiy Storchaka in branch 'master':
bpo-29922: Improve error messages in 'async with' (GH-6352)
https://github.com/python/cpython/commit/a68f2f0578bbf812fa2264d0e0bb388340d6e230
msg314838 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-04-02 22:51
It would be hard to backport this to 3.5. Wordcode allows scanning back.
msg314839 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-04-02 22:56
3.5 is in security fixes only, so don't bother.
msg314932 - (view) Author: miss-islington (miss-islington) Date: 2018-04-04 14:09
New changeset fcd4e03e08a2d4ec1cde17beb66e2b22a052500f by Miss Islington (bot) in branch '3.7':
bpo-29922: Improve error messages in 'async with' (GH-6352)
https://github.com/python/cpython/commit/fcd4e03e08a2d4ec1cde17beb66e2b22a052500f
msg314933 - (view) Author: miss-islington (miss-islington) Date: 2018-04-04 14:09
New changeset 4fd6c27dc8ba7ca97aa70e1ab98729f2207bbe19 by Miss Islington (bot) in branch '3.6':
bpo-29922: Improve error messages in 'async with' (GH-6352)
https://github.com/python/cpython/commit/4fd6c27dc8ba7ca97aa70e1ab98729f2207bbe19
msg314939 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-04-04 15:45
New changeset 2eeac269dd1e04a2a179384576986c3e47895ee0 by Serhiy Storchaka in branch 'master':
bpo-29922: Add more tests for error messages in 'async with'. (GH-6370)
https://github.com/python/cpython/commit/2eeac269dd1e04a2a179384576986c3e47895ee0
msg314940 - (view) Author: miss-islington (miss-islington) Date: 2018-04-04 16:28
New changeset 785f36c876721c12f653371e9893527a25140624 by Miss Islington (bot) in branch '3.7':
bpo-29922: Add more tests for error messages in 'async with'. (GH-6370)
https://github.com/python/cpython/commit/785f36c876721c12f653371e9893527a25140624
msg314941 - (view) Author: miss-islington (miss-islington) Date: 2018-04-04 17:18
New changeset 1487cd14bb1e04de5b98fffc5ec41c6cf6b5d5f1 by Miss Islington (bot) in branch '3.6':
bpo-29922: Add more tests for error messages in 'async with'. (GH-6370)
https://github.com/python/cpython/commit/1487cd14bb1e04de5b98fffc5ec41c6cf6b5d5f1
msg314944 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-04-04 17:39
Since BEFORE_ASYNC_WITH always is followed by GET_AWAITABLE, in future they can be merged into a single instruction (like GET_ANEXT or GET_YIELD_FROM_ITER).
History
Date User Action Args
2018-04-04 17:39:22serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg314944

stage: patch review -> resolved
2018-04-04 17:18:59miss-islingtonsetmessages: + msg314941
2018-04-04 16:30:17miss-islingtonsetpull_requests: + pull_request6084
2018-04-04 16:28:51miss-islingtonsetmessages: + msg314940
2018-04-04 15:46:30miss-islingtonsetpull_requests: + pull_request6082
2018-04-04 15:45:20serhiy.storchakasetmessages: + msg314939
2018-04-04 14:09:17miss-islingtonsetmessages: + msg314933
2018-04-04 14:09:06miss-islingtonsetnosy: + miss-islington
messages: + msg314932
2018-04-04 14:08:30serhiy.storchakasetpull_requests: + pull_request6081
2018-04-02 22:56:38yselivanovsetmessages: + msg314839
2018-04-02 22:51:09serhiy.storchakasetmessages: + msg314838
2018-04-02 22:42:48miss-islingtonsetpull_requests: + pull_request6063
2018-04-02 22:42:04miss-islingtonsetpull_requests: + pull_request6062
2018-04-02 22:41:48serhiy.storchakasetmessages: + msg314836
2018-04-02 22:31:43ned.deilysetmessages: + msg314834
versions: + Python 3.6
2018-04-02 22:28:04yselivanovsetmessages: + msg314832
2018-04-02 22:23:53ned.deilysetmessages: + msg314831
versions: + Python 3.7
2018-04-02 21:48:36serhiy.storchakasetnosy: + ned.deily, serhiy.storchaka, Mark.Shannon
superseder: Traceback from __exit__ method is misleading ->
messages: + msg314827
2018-04-02 21:36:49serhiy.storchakasetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request6061
2017-12-20 20:05:25yselivanovsetversions: + Python 3.8, - Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7
2017-12-20 20:02:43asvetlovsetnosy: + asvetlov
messages: + msg308797
components: + Interpreter Core, - asyncio
2017-03-31 19:53:35levkivskyisetnosy: + levkivskyi
2017-03-29 08:43:18RazerMsetnosy: + RazerM
2017-03-28 07:40:47Chewysetnosy: + Chewy
2017-03-27 19:56:48yselivanovsetstatus: closed -> open
resolution: duplicate -> (no value)
stage: resolved -> needs patch
2017-03-27 19:56:30yselivanovsetmessages: + msg290640
2017-03-27 19:53:36Tadhg McDonald-Jensensetmessages: + msg290639
2017-03-27 17:46:10yselivanovsetassignee: yselivanov
messages: + msg290633
2017-03-27 17:33:56r.david.murraysetstatus: open -> closed

superseder: Traceback from __exit__ method is misleading

nosy: + r.david.murray
messages: + msg290631
resolution: duplicate
stage: resolved
2017-03-27 17:24:22Tadhg McDonald-Jensencreate