classification
Title: Dynamic compilation that uses function in comprehension fails when compiled inside function
Type: behavior Stage: resolved
Components: Versions: Python 3.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: jpc, r.david.murray, veky
Priority: normal Keywords:

Created on 2017-09-08 21:19 by jpc, last changed 2017-09-12 13:51 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
reproduce.zip jpc, 2017-09-08 21:19 Zip file contaIning three python programs to reproduce error
Messages (7)
msg301734 - (view) Author: Jose Cambronero (jpc) Date: 2017-09-08 21:19
Execution fails with NameError when trying to dynamically compile a Python program that uses a function in a comprehension, and the code is compiled inside a function call, rather than as a global statement.

Using the attached files to reproduce (in reproduce.zip):

python3 compile_in_function.py comprehension.py #fails
python3 compile_global.py comprehension.py # works
python2 compile_in_function.py comprehension.py #works with Python 2.7.12
msg301784 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-09-09 17:39
Could you please post the examples instead of a zip file?  Zip files are hard to read on a mobile browser :)
msg301790 - (view) Author: Jose Cambronero (jpc) Date: 2017-09-09 20:32
Woops, sorry about that, makes sense. Below an example (same idea as the files):

```
Python 3.6.2 (default, Sep  9 2017, 13:27:06) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> src = """
... def f(x): return x
... [f(v) for v in [1,2,3]]
... """
>>> def comp():
...     exec(compile(src, filename='<str>', mode='exec'))
... 
>>> comp()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in comp
  File "<str>", line 3, in <module>
  File "<str>", line 3, in <listcomp>
NameError: name 'f' is not defined
>>> exec(compile(src, filename='<str>', mode='exec'))
>>> 
```
msg301794 - (view) Author: Vedran Čačić (veky) * Date: 2017-09-09 23:14
List comprehension is irrelevant here, any nested function has the same problem. And the inner name doesn't have to refer to a function. Also, compilation as a separate step is not necessary.

So the minimal example is

    src = '''\
    a = 5
    def g(x): return a + x
    print(g(3))
    '''
    def comp(): exec(src)
    comp()

But I'm not sure if it is a bug at all. If you write explicitly

    exec(src, {})

you get a result 8 without problem. The docs say

    ...if the optional parts are omitted, the code is executed in the current scope.

And the current scope inside comp is "fast locals", where you can't really make new names. Maybe exec should raise an exception then, but it surely cannot just "work" inside fast locals scope.
msg301795 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-09-09 23:19
This is indeed not a bug, it it a consequence of the scoping rules.

What surprises me is that it works without passing in an explicit scope in in 3.7.  I'm not sure what we changed that makes that true.
msg301938 - (view) Author: Vedran Čačić (veky) * Date: 2017-09-12 05:55
Hm... then it _is_ a bug (in 3.7). Or we have "fast locals and slow locals" nightmare again... :-/

I don't have 3.7 to test... what do you mean is working? jpc's example or mine?
msg301961 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-09-12 13:51
Hmm.  I must have made a mistake when I ran (jpc's) test on 3.7.  It is failing with the NameError for me when I try it again.
History
Date User Action Args
2017-09-12 13:51:35r.david.murraysetstatus: open -> closed
type: compile error -> behavior
messages: + msg301961

resolution: not a bug
stage: resolved
2017-09-12 05:55:33vekysetmessages: + msg301938
2017-09-09 23:19:14r.david.murraysetmessages: + msg301795
2017-09-09 23:14:34vekysetnosy: + veky
messages: + msg301794
2017-09-09 20:32:24jpcsetmessages: + msg301790
2017-09-09 17:39:18r.david.murraysetnosy: + r.david.murray
messages: + msg301784
2017-09-08 21:19:38jpccreate