classification
Title: [doc] Unintuitive error when using generator expression in class property
Type: behavior Stage: needs patch
Components: Documentation Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: abarry, corey, docs@python, mark.dickinson, serhiy.storchaka, steven.daprano, xiang.zhang
Priority: normal Keywords:

Created on 2016-05-04 14:34 by corey, last changed 2021-12-12 00:51 by iritkatriel.

Messages (5)
msg264819 - (view) Author: Corey Farwell (corey) Date: 2016-05-04 14:34
```
class A:
    B = range(10)
    C = frozenset([4, 5, 6])
    D = list(i for i in B)
    E = list(i for i in B if i in C)
```

```

coreyf@frewbook-pro /tmp [1]> python3 a.py
Traceback (most recent call last):
  File "a.py", line 1, in <module>
    class A:
  File "a.py", line 5, in A
    E = list(i for i in B if i in C)
  File "a.py", line 5, in <genexpr>
    E = list(i for i in B if i in C)
NameError: name 'C' is not defined
```

Why should I be able to access B but not C?
msg264821 - (view) Author: Anilyka Barry (abarry) * (Python triager) Date: 2016-05-04 15:00
Using a simple metaclass shows that definition order is not at fault here:

>>> class PrintOrder(dict):
...   def __setitem__(self, item, value):
...     print(item, value)
...     super().__setitem__(item, value)
...
>>> class Show(type):
...   def __prepare__(name, bases): return PrintOrder()
...
>>> class A(metaclass=Show):
...   B = range(10)
...   C = frozenset([4, 5, 6])
...   D = list(i for i in B)
...   E = list(i for i in B if i in C)
...
__module__ __main__
__qualname__ A
B range(0, 10)
C frozenset({4, 5, 6})
D [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in A
  File "<stdin>", line 5, in <genexpr>
NameError: name 'C' is not defined

However, the following works:

>>> def local_definition():
...   F = frozenset([4, 5, 6])
...   class A(metaclass=Show):
...     B = range(10)
...     C = frozenset([4, 5, 6])
...     D = list(i for i in B)
...     E = list(i for i in B if i in F)
...   return A
...
>>> local_definition()
__module__ __main__
__qualname__ local_definition.<locals>.A
B range(0, 10)
C frozenset({4, 5, 6})
D [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
E [4, 5, 6]
<class '__main__.local_definition.<locals>.A'>

Sounds like either an inconsistency between 'for' and 'if' parts of generator expressions, or something weird about the way generator expressions work that I'm unaware of.

Furthermore, this isn't documented in the tutorial, the functional programming How-To, the 2.4 release notes or the PEP 289:

https://docs.python.org/3/tutorial/classes.html#generator-expressions
https://docs.python.org/3/howto/functional.html#generator-expressions-and-list-comprehensions
https://docs.python.org/3/whatsnew/2.4.html#pep-289-generator-expressions
https://www.python.org/dev/peps/pep-0289/
msg264830 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2016-05-04 15:54
On snap! Nicely found! This seems to have something to do with the way generator expressions are run inside their own scope.

In Python 2.7, a list comprehension in a class sees the locals correctly:

py> class K:
...     a = 2; x = [a+i for i in range(a)]
...
py> K.x
[2, 3]


But change the list comp to a generator expression, and the situation is different:


py> class K:
...     a = 2; x = list(i for i in range(a))  # Okay
...     b = 2; y = list(b+i for i in range(a))  # Raises
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in K
  File "<stdin>", line 3, in <genexpr>
NameError: global name 'b' is not defined


In Python 3, list comps use the same sort of temporary scope as genexprs, and sure enough, the list comp fails with the same error as the generator expression.
msg264845 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-05-04 18:31
The outer for loop in a generator expression is evaluated immediately; everything after that is evaluated lazily. This was a deliberate design choice: see https://www.python.org/dev/peps/pep-0289/#early-binding-versus-late-binding for some rationale. This explains why you're able to access "B". The inability to access "C" is due to the usual rule for Python scope resolution: class scopes are skipped when examining nested scopes.

The early evaluation of the outermost for is covered in the docs, here: https://docs.python.org/2/reference/expressions.html#generator-expressions

"""
However, the leftmost for clause is immediately evaluated, so that an error produced by it can be seen before any other possible error in the code that handles the generator expression.
"""

So this behaviour is by design, though there may be a doc issue here. Changing components and versions accordingly.
msg348191 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-07-19 18:39
See also issue3692.
History
Date User Action Args
2021-12-12 00:51:42iritkatrielsettitle: Unintuitive error when using generator expression in class property -> [doc] Unintuitive error when using generator expression in class property
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 2.7, Python 3.5, Python 3.6
2019-07-19 19:00:22serhiy.storchakalinkissue35625 superseder
2019-07-19 18:39:56serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg348191
2016-05-04 18:31:34mark.dickinsonsetversions: + Python 3.6, - Python 3.2, Python 3.3, Python 3.4
nosy: + mark.dickinson, docs@python

messages: + msg264845

assignee: docs@python
components: + Documentation, - Interpreter Core
2016-05-04 16:01:45xiang.zhangsetnosy: + xiang.zhang
2016-05-04 15:54:43steven.dapranosetnosy: + steven.daprano
messages: + msg264830
2016-05-04 15:00:56abarrysetnosy: + abarry

messages: + msg264821
stage: needs patch
2016-05-04 14:34:33coreycreate