classification
Title: Strange behavior with await in a generator expression
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: levkivskyi, yselivanov
Priority: normal Keywords:

Created on 2017-11-22 14:42 by levkivskyi, last changed 2020-07-25 22:43 by Bryan Hu.

Messages (3)
msg306732 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2017-11-22 14:42
PEP 530 is not very clear about `await` in generator expressions. But when I try it, the error is a bit confusing:

>>> async def g(i):
...     print(i)
... 
>>> async def f():
...     result = list(await g(i) for i in range(3))
...     print(result)
... 
>>> f().send(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in f
TypeError: 'async_generator' object is not iterable

At the same time a (seemingly) equivalent list comprehension works fine:

>>> async def f():
...     result = [await g(i) for i in range(3)]
...     print(result)
... 
>>> f().send(None)
0
1
2
[None, None, None]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

I would say that the first case should either behave as a second one, or raise a syntax error.

Or is it actually an intended behavior?
msg306733 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2017-11-22 15:46
> ...     result = list(await g(i) for i in range(3))

This is equivalent to this code:

  async def ait():
      for i in range(3):
          v = await g(i)
          yield v

  result = list(ait())

Where 'ait' is an async generator function.  You can't iterate it with the regular 'for x in ...' syntax, and you can't pass it to functions that expect a synchronous iterator (such as 'list').

Similarly, with synchronous code:

  a = (i for i in range(3))
  a[0]
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  TypeError: 'generator' object is not subscriptable

where '(' for ... ')' is another syntax for defining a synchronous generator.


> ...     result = [await g(i) for i in range(3)]

This is equivalent to this code:

  result = []
  for i in range(3):
      v = await g(i)
      result.append(v)


I agree that PEP 530 is a bit vague about this and can be updated.  I'll take a look into that.

Perhaps we can make the "TypeError: 'async_generator' object is not iterable" error message a bit clearer.  Any ideas to improve it are welcome.

> I would say that the first case should either behave as a second one, or raise a syntax error.

No, but we can improve error messages.
msg306736 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2017-11-22 16:50
A first simple idea that comes to my mind is special-case async generators/iterators in PyObject_GetIter to say something like:

TypeError: asynchronous iterable can't be used where an iterable is expected

If it is possible to detect that an async generator is resulting from a generator expression, then we can say:

TypeError: asynchronous generator expression can't be used as an iterable
History
Date User Action Args
2020-07-25 22:43:12Bryan Husetversions: + Python 3.8
2017-11-22 16:50:44levkivskyisetmessages: + msg306736
2017-11-22 15:46:58yselivanovsetmessages: + msg306733
2017-11-22 14:42:41levkivskyicreate