classification
Title: New misleading wording in execution model documenation
Type: enhancement Stage:
Components: Documentation Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: abarnert, docs@python, eryksun, martin.panter
Priority: normal Keywords:

Created on 2016-01-28 00:01 by abarnert, last changed 2016-01-30 05:14 by martin.panter.

Messages (3)
msg259073 - (view) Author: Andrew Barnert (abarnert) * Date: 2016-01-28 00:01
In #24129, the wording describing class local bindings in 4.2.2 "Resolution of names" was changed for Python 3.4, 3.5, and 3.6. The new version is a lot clearer for classes--but now it's misleading for `exec`/`eval`.

---

> Class definition blocks and arguments to exec() and eval() are
> special in the context of name resolution. A class definition is...

... and then proceeds to explain how class lookup works, without ever mentioning `exec` and `eval`. This implies that they work the same way as classes, but of course that's not true:

    i = 'global'
    def f():
        i = 'nonlocal'
        class C:
            print(i)
            i = 'local'
            print(i)
    f()

That prints `global`, then `local`. But with `exec`:

    i = 'global'
    def f():
        i = 'nonlocal'
        exec("print(i)\ni = 'local'\nprint(i)\n")
    f()

That prints `nonlocal` then `local`.

I think just putting a paragraph break between the first sentence and the rest of the paragraph might be sufficient to avoid the confusion here. Or just removing any mention of `eval` and `exec`. If not, this probably needs a new one-liner paragraph saying something like "Arguments to `exec()` and `eval()` are also special, as described later."

---

Meanwhile, if you keep reading, you'll eventually find that `exec` is described in a later section, 4.2.4 "Interaction with dynamic features", but that's _also_ misleading:

> The eval() and exec() functions do not have access to the full
> environment for resolving names. Names may be resolved in the
> local and global namespaces of the caller. Free variables are not
> resolved in the nearest enclosing namespace, but in the global
> namespace.

If that were true, the `exec` example would have printed `global`, right?

I'm pretty sure that what's going on here is that `exec` implicitly calls `locals()` (or, rather, the C-API equivalent), which constructs a locals dict on demand, which, only if you're inside a function block, includes not just the currently-bound fast locals, but _also_ the cell_contents of the currently-bound free variables. So, as far as `exec` is concerned, `i` is not an unbound local, or a free variable, but a local, which is bound to the `'nonlocal'` cell value of `i` at the time `exec` was called.

Which means the following actually _does_ print `global`:

    i = 'global'
    def f():
        exec("print(i)\ni = 'local'\nprint(i)\n")
        i = 'nonlocal'
    f()

I have no idea how to make this clear. Maybe the simplest is to not try to give a full explanation here, and instead punt to the `locals()` function definition? Maybe something like this:

> The `eval()` and `exec()` functions do not have access to the full environment for resolving names, but rather to the approximation of that environment as constructed by the `locals()` function. Free variables that are not captured as locals are not resolved in the nearest enclosing namespace, but in the global...

... and from there, the same as the current paragraph.
msg259083 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2016-01-28 01:29
The class example defines "i" as a local variable, which means the CPython operation used for unoptimized code (class or module/exec) is LOAD_NAME, which searches locals, globals, and builtins. The result differs from the exec example because a class is executed with a new locals dict to capture the class namespace. 

I think a more interesting case to explain is code that uses LOAD_CLASSDEREF. This operation tries locals and nonlocals, but not globals or builtins.

    i = 'global'
    def f():
        i = 'nonlocal'
        class C:
            print(i)

    >>> f()
    nonlocal

    i = 'global'
    def f():
        class C:
            print(i)
        i = 'nonlocal'

    >>> f()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<stdin>", line 2, in f
      File "<stdin>", line 3, in C
    NameError: free variable 'i' referenced before assignment in enclosing scope

    i = 'global'
    def f():
        class C:
            locals()['i'] = 'local'
            print(i)
        i = 'nonlocal'

    >>> f()
    local

    i = 'global'
    def f():
        i = 'nonlocal'
        class C:
            nonlocal i
            print(i)
            i = 'new nonlocal'
            print(i)
        print(i)

    >>> f()
    nonlocal
    new nonlocal
    new nonlocal
msg259237 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-01-30 05:14
Calling exec() with only one argument is equivalent to exec(..., globals(), locals()). It does not create a new scope for names. So an equivalent of your three-level example is more like

>>> i = 'global'
>>> def f():
...     i = 'nonlocal'
...     class_locals = dict()
...     exec("print(i)\ni = 'local'\nprint(i)\n", globals(), class_locals)
... 
>>> f()
global
local

If exec() worked like a function rather than a class, the first print(i) would trigger an UnboundLocalError instead:

>>> i = 'global'
>>> def f():
...     i = 'nonlocal'
...     def g():
...         print(i)
...         i = 'local'
...     g()
... 
>>> f()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 6, in f
  File "<stdin>", line 4, in g
UnboundLocalError: local variable 'i' referenced before assignment

In your first exec() example, i='nonlocal' is passed to exec() via the default locals parameter, and the exec() uses that value rather than deferring to its globals. To be a free variable, “i” has to be used but not defined. Even if you dropped the “i = 'local' ” assignment, it is still defined via the implicit locals parameter.

Your proposal for “Interaction with dynamic features” sounds reasonable.
History
Date User Action Args
2016-01-30 05:14:19martin.pantersetnosy: + martin.panter
messages: + msg259237
2016-01-28 01:29:15eryksunsetnosy: + eryksun
messages: + msg259083
2016-01-28 00:01:40abarnertsettype: enhancement
2016-01-28 00:01:28abarnertcreate