This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: PyEval_EvalCode() namespace issue not observed in Python 2.7.
Type: behavior Stage: resolved
Components: C API Versions: Python 3.8
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: chrisgmorton, terry.reedy
Priority: normal Keywords:

Created on 2021-03-12 21:44 by chrisgmorton, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (7)
msg388557 - (view) Author: Chris Morton (chrisgmorton) Date: 2021-03-12 21:44
Compiling (Window 10, MSVS 16):

#include <Python.h>

int main(int argc, char* argv[])
{
    const char* code = "c=[1,2,3,4]\nd={'list': [c[i] for i in range(len(c))]}\nprint(d)\n";

    Py_Initialize();

    PyObject* pycode = Py_CompileString(code, "", Py_file_input );   
    PyObject* main_module = PyImport_AddModule("__main__");
    PyObject* global_dict = PyModule_GetDict(main_module);
    PyObject* local_dict = PyDict_New();
    
    PyEval_EvalCode(pycode, global_dict, local_dict);  // (PyCodeObject*) pycode in Python 2.7
    
    Py_Finalize();

    return 0;

and executing yields:

Traceback (most recent call last):
  File "", line 2, in <module>
  File "", line 2, in <listcomp>
NameError: name 'c' is not defined

While not particularly clever python code, it is not clear why the reference c is not in scope having previously been defined. Replacing the clumsy list comprehension using range() with c[:] or [ci for ci in c] produces the expected result:

{'list': [1, 2, 3, 4]}

This issue is not observed with Python 2.7 (.18).
msg388852 - (view) Author: Chris Morton (chrisgmorton) Date: 2021-03-16 16:09
Root cause appears to be indexing c with [] operator. Replacing len(c) with 4 produces the same error.
msg389120 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-03-19 22:38
I cannot reproduce in Python with either 3.8 or 3.10.  (Please try with latter if you can.)  I thought the issue might possibly be passing two different dicts, which results in the code being executed as if in a class statement, but it is not.  

code = '''
c=[1,2,3,4]
d={'list': [c[i] for i in range(len(c))]}
print(d)
'''
bcode = compile(code, '', 'exec')
gdict = globals()
ldict = {}
exec(bcode, gdict, gdict)
exec(bcode, gdict, ldict)
class C:
    c=[1,2,3,4]
    d={'list': [c[i] for i in range(len(c))]}
    print(d)

prints {'list': [1, 2, 3, 4]} 3 times.  Using 'eval' instead of 'exec' gives the same.  I presume that code compiled with 'exec' is 'exec'ed even if use eval.
msg389165 - (view) Author: Chris Morton (chrisgmorton) Date: 2021-03-20 15:29
Hi Terry, The reason why your code does not reproduce the issue is because you first execute the code in a global context which then puts the definition of c in that context. Subsequent calling in the local context then works. If you remove the first exec call (no need for the Class lines either), you will now see the issue in 3.8.
This issue is reproducible for other containers such as lists and sets, instead of dictionary, in this case. Replacing range(len(c)) with range(4) also shows the same error. The error relates to the use of the indexing operator []. The same error is observed with other sequences such as:

c={1,2,3,4} 

or

c='1234'
msg389178 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-03-20 20:29
Foolish me. Commenting out the first exec results in the 2nd exec raising.  Commenting out the 2nd exec also results in the class code raising, which is what I expected.  The point of the class code is to partially explain the exception, which is not a bug, but a consequence of passing separate global and initial local namespaces, combined with comprehensions being executed, starting in 3.0, in a new local namespace, separate from the one you passed in.

The exec doc in https://docs.python.org/3/library/functions.html#exec explains the effect of passing different global and local namespaces with "If exec gets two separate objects as globals and locals, the code will be executed as if it were embedded in a class definition."  There is no class, but this is the only way to get the same effect in Python code (without exec/eval).

The NameError has nothing to do with subscripting as it happens upon the attempt to load c.  Remove the subscription and the exception it remains.  Instead, it arises because in 3.x, but not in 2.x, comprehensions are executed in a separate local namespace, the same as a method, where the passed in local namespace is invisible.

Here are simpler examples that both raise NameError.

exec("c=1\nlist(c for i in [1])", globals(), {})
----

class A:
    c =1
    def f(self): c

A().f()
msg389468 - (view) Author: Chris Morton (chrisgmorton) Date: 2021-03-24 15:20
Is this change in behavior considered to be by design or is this issue an unintended consequence? It seems inconsistent that comprehensions have no visibility of variable previously defined in the same scoping for this particular use-case. Is there a way to work around this other than reworking the comprehension?

It seems odd that [ci for ci in c] does not produce the error but [c for i in [1]] does produce the error.
msg389475 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-03-24 16:59
Perhaps both.  If you want more discussion, please post to python-list.
History
Date User Action Args
2022-04-11 14:59:42adminsetgithub: 87647
2021-03-24 16:59:06terry.reedysetmessages: + msg389475
2021-03-24 15:20:11chrisgmortonsetmessages: + msg389468
2021-03-20 20:29:23terry.reedysetstatus: open -> closed
type: behavior
messages: + msg389178

resolution: not a bug
stage: resolved
2021-03-20 15:29:56chrisgmortonsetmessages: + msg389165
2021-03-19 22:38:53terry.reedysetnosy: + terry.reedy
messages: + msg389120
2021-03-16 16:09:59chrisgmortonsetmessages: + msg388852
2021-03-12 21:44:37chrisgmortoncreate