The following one-liner works fine in a regular Python interpreter:

$ python -c 'import sys; x = 5; [print(x + i) for i in range(5)]'
5
6
7
8
9

But in a .pth file, it raises a NameError:

$ echo 'import sys; x = 5; [print(x + i) for i in range(5)]' | sudo tee /usr/lib/python3.8/site-packages/test.pth
$ python
Error processing line 1 of /usr/lib/python3.8/site-packages/test.pth:

  Traceback (most recent call last):
    File "/usr/lib/python3.8/site.py", line 169, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
    File "<string>", line 1, in <listcomp>
  NameError: name 'x' is not defined

Remainder of file ignored
Python 3.8.0 (default, Oct 23 2019, 18:51:26) 
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Since site.py uses exec() to exec each line of a .pth file, I thought I'd compare with that. It also works fine:

$ python -c 'exec("import sys; x = 5; [print(x + i) for i in range(5)]")'
5
6
7
8
9

This slight modification (the variable being used in the next statement still, but not within the loop of the comprehension) does not raise a NameError:

$ echo 'import sys; x = 5; [print(i) for i in range(x)]' | sudo tee /usr/lib/python3.8/site-packages/test.pth
import sys; x = 5; [print(i) for i in range(x)]
$ python
0
1
2
3
4
Python 3.8.0 (default, Oct 23 2019, 18:51:26) 
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

I know .pth file processing is very early in interpreter startup such that many things aren't working yet, but I wouldn't expect using a name defined outside a list comprehension within the loop body of said list comprehension not to work.

The following is fine also, showing that using names from outside the list comprehension doesn't always break:

$ echo 'import sys; [print(sys) for i in range(5)]' | sudo tee /usr/lib/python3.8/site-packages/test.pth
import sys; [print(sys) for i in range(5)]
$ python
<module 'sys' (built-in)>
<module 'sys' (built-in)>
<module 'sys' (built-in)>
<module 'sys' (built-in)>
<module 'sys' (built-in)>
Python 3.8.0 (default, Oct 23 2019, 18:51:26) 
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

This is fine too:

$ echo 'import sys; [print(str(sys) * i) for i in range(5)]' | sudo tee /usr/lib/python3.8/site-packages/test.pth
import sys; [print(str(sys) * i) for i in range(5)]
$ python

<module 'sys' (built-in)>
<module 'sys' (built-in)><module 'sys' (built-in)>
<module 'sys' (built-in)><module 'sys' (built-in)><module 'sys' (built-in)>
<module 'sys' (built-in)><module 'sys' (built-in)><module 'sys' (built-in)><module 'sys' (built-in)>
Python 3.8.0 (default, Oct 23 2019, 18:51:26) 
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 


My use case is looping over subdirs of a directory and adding them all to sys.path to provide similar functionality to python setup.py develop, with all python vcs repositories within a specific directory being prepended to sys.path, rather than having to add them one-by-one. I probably won't end up doing what I'm doing this way, but in any case the above seems like it's a bug, unless I'm grossly misunderstanding something.

I see. site.py calls exec() from within a function, and therefore the code is executed in the context of that function's locals and the site module globals. This means the code in .pth files can access (but not add new) local names from the site.addpackage() function:

$ echo 'import sys; f.close()' | sudo tee /usr/lib/python3.8/site-packages/test.pth
import sys; f.close()
$ python
Fatal Python error: init_import_size: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
  File "/usr/lib/python3.8/site.py", line 580, in <module>
    main()
  File "/usr/lib/python3.8/site.py", line 567, in main
    known_paths = addsitepackages(known_paths)
  File "/usr/lib/python3.8/site.py", line 350, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "/usr/lib/python3.8/site.py", line 208, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "/usr/lib/python3.8/site.py", line 164, in addpackage
    for n, line in enumerate(f):
ValueError: I/O operation on closed file.

The example with the sys module worked because sys is in the globals the site module already.

Probably site.addpackage() should exec() code it its own environment:

if line.startswith(("import ", "import\t")):
    exec(line, {})
    continue

(added empty dict for exec() call)

or for backward compatibility for .pth files that are using globals from the site module without importing them (such as sys or os):

if line.startswith(("import ", "import\t")):
    exec(line, globals().copy())
    continue

This resolves the original issue

Sorry for the spamming, realised I misunderstood further.

The original behaviour isn't because the exec'd code can't create new local variables - it can - it's because of the documented behaviour of exec when it gets different dicts for globals and locals:

"If exec gets two separate objects as globals and locals, the code will be executed as if it were embedded in a class definition"

So the new scope made by the list comprehension can't access the enclosing scope in which the new variable was defined, because that's how scoping works in class definitions.