Creating a venv from the python.exe from another venv does not work since 3.7.2 on Windows. This is probably related to the change

bpo-34977: venv on Windows will now use a python.exe redirector rather than copying the actual binaries from the base environment.

For example running

c:\users\$USER\.local\pipx\venvs\pipx-app\scripts\python.exe -m venv C:\Users\$USER\.local\pipx\venvs\tox
C:\Users\$USER\.local\pipx\venvs\tox\Scripts\python.exe -m pip install --upgrade pip

results in pip installing to venvs\pipx-app and not in venvs\tox.

Downstream bugreport in pipx is pypa/pipx#81.

The __PYVENV_LAUNCHER__ variable gets set by each launcher instance. We get one launcher process for every level of nesting, and the last launcher to run sets the final value that will be seen by the real python.exe. Possibly the launcher could resolve the home values until it reaches a solid python.exe (i.e. no pyvenv.cfg file), and thus avoid the nested chain of launcher processes and ensure that the real python.exe sees the correct value of __PYVENV_LAUNCHER__.

This scenario should work, as running the other venv's redirector will update the variable.

The nearly identical report in bpo-35873 is apparently launching "default" Python, but presumably without clearing the variable. Since I can't tell what's going wrong with the report in this bug, I'm going to focus on the other one.

This scenario should work, as running the other venv's redirector
will update the variable.

The order gets reversed. In the simple case where we have two launchers, the launcher for the nested virtual environment executes the launcher for the outer (creator) virtual environment, which executes the real python.exe. So Python sees the wrong value for __PYVENV_LAUNCHER__. For example, if I create env37_2 from env37_1, here's the result in env37_2:

    C:\>C:\Temp\test\env37_2\scripts\python.exe
    Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28)
    [MSC v.1916 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sys, os
    >>> print(sys.executable)
    C:\Temp\Test\env37_1\scripts\python.exe

>>> print(*sys.path, sep='\n')

C:\Program Files\Python37\python37.zip
C:\Program Files\Python37\DLLs
C:\Program Files\Python37\lib
C:\Program Files\Python37
C:\Temp\Test\env37_1
C:\Temp\Test\env37_1\lib\site-packages

    >>> print(os.environ['__PYVENV_LAUNCHER__'])
    C:\Temp\Test\env37_1\scripts\python.exe

You can't actually nest virtual environments, at least with venv - it ought to be configuring it all against the original environment.

What is in your pyvenv.cfg files here?

You can't actually nest virtual environments, at least with venv

Yes, they can't be nested in terms of inheriting site-packages. But the "home" value in this case references the creating environment, which hasn't changed from previous versions. It's the same in Linux. In my first message I suggested having the launcher resolve pyvenv.cfg files until it finds the real python.exe to execute (the first one with no pyvenv.cfg file), thus bypassing the intervening launchers.

What is in your pyvenv.cfg files here?

They're as expected:

C:\>type C:\Temp\test\env37_1\pyvenv.cfg
home = C:\Program Files\Python37
include-system-site-packages = false
version = 3.7.2

C:\>type C:\Temp\test\env37_2\pyvenv.cfg
home = C:\Temp\Test\env37_1\scripts
include-system-site-packages = false
version = 3.7.2

Is there any reason to not resolve the base executable on creation and make it relative to that? So the second pyvenv.cfg would have the same home directory as the first?

My proposed fix for bpo-35873 (PR soon) is going to make this trivial.

Is there any reason to not resolve the base executable on creation and
make it relative to that? So the second pyvenv.cfg would have the same
home directory as the first?

I was trying to avoid changing the existing behavior of home from how it works in 3.6. Your solution will introduce a (probably minor) change, which may be for the best anyway. The computed argv0_path gets added to sys.path, which in the normal case is the installation directory, but it gets overridden by home. In 3.6, in the nested-create case, the argv0_path is the creating environment's Scripts directory. For example:

C:\>type C:\Temp\test\env36_2\pyvenv.cfg
home = C:\Temp\Test\env36_1\scripts
include-system-site-packages = false
version = 3.6.4

    C:\>C:\Temp\test\env36_2\Scripts\python.exe -S
    Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40)
    [MSC v.1900 64 bit (AMD64)] on win32
    >>> import sys
    >>> print(*sys.path, sep='\n')

C:\Temp\test\env36_2\Scripts\python36.zip
C:\Program Files\Python36\Lib\
C:\Program Files\Python36\DLLs\
C:\Temp\Test\env36_1\scripts

I don't know why we add argv0_path to sys.path. If it's meant to be the directory of the (not necessarily real) executable, then the last path in the above example should be "C:\Temp\Test\env36_2\Scripts" instead of "env36_1\Scripts". If the argv0_path is meant to be the installation directory, as part of the standard library, then it should be "C:\Program Files\Python36". PR 11745 will make it consistently the latter for all virtual environments.

I don't know why we add argv0_path to sys.path.

My guess is that the idea was to be able to load the .pyd files we used to put alongside it, though we found out they weren't being loaded anyway.

One potential side effect is that manually created .pth files may no longer have an effect, as those likely chain together just fine. But I don't think that's a likely scenario.

New changeset a8474d0 by Steve Dower in branch 'master':
bpo-35872 and bpo-35873: Clears __PYVENV_LAUNCHER__ variable (GH-11745)
a8474d0

New changeset 44467e8 by Steve Dower in branch '3.7':
bpo-35872 and bpo-35873: Clears __PYVENV_LAUNCHER__ variable (GH-11745)
44467e8

See pypa/virtualenv#1339 - it's possible that something in this area is still affecting virtualenv.