I upgraded from Python 3.7.1 to 3.7.6 and began noticing a behavior that was breaking my code. My code detects if it's running in a virtualenv. This check worked in 3.7.1 but is broken in 3.7.6.

From the documentation, sys.prefix and sys.exec_prefix should point to the virtualenv when one is active. However, I'm seeing that both of these constants are pointing to the system installation directory and not my virtualenv when I am in a multiprocessing child. Here is an example output of a test application running in 3.7.6 (I've attached the test script to this ticket):

Parent process
=============================================
sys.prefix: C:\Users\user\project\venv
sys.exec_prefix: C:\Users\user\project\venv
sys.base_prefix: C:\Program Files\Python37
sys.base_exec_prefix: C:\Program Files\Python37
=============================================

Subprocess
=============================================
sys.prefix: C:\Program Files\Python37
sys.exec_prefix: C:\Program Files\Python37
sys.base_prefix: C:\Program Files\Python37
sys.base_exec_prefix: C:\Program Files\Python37
=============================================

I would expect that sys.prefix and sys.exec_prefix to be identical in the parent and child process.

I verified that this behavior is present in 3.7.5, 3.7.6, and 3.8.1. I am on a Windows 10 x64 system.

Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)] on win32

This is a duplicate of bpo-38092. The required change to fix this is very simple, if you want to work on a PR.

OK, that makes sense.

For 3.7, I can create a PR for that corrects the order of arguments passed into _winapi.CreateProcess

For 3.8 / master, the problem appears to be that the check in popen_spawn_win32.py to set the subprocess env is failing because sys.executable != spawn.get_executable() -- spawn.get_executable() is returning sys._base_executable. So, can you confirm that the fix is to just change spawn.get_executable() to return sys.executable, like it was prior to the PR mentioned in the other ticket?

can you confirm that the fix is to just change spawn.get_executable()
to return sys.executable, like it was prior to the PR mentioned in
the other ticket?

Yes, in spawn.py, change _python_exe = sys._base_executable to _python_exe = sys.executable. This is the value that gets returned by spawn.get_executable().

This will restore the operation of the following code in Popen.__init__ in popen_spawn_win32.py:

        python_exe = spawn.get_executable()

        # bpo-35797: When running in a venv, we bypass the redirect
        # executor and launch our base Python.
        if WINENV and _path_eq(python_exe, sys.executable):
            python_exe = sys._base_executable
            env = os.environ.copy()
            env["__PYVENV_LAUNCHER__"] = sys.executable

This code switches to running the base executable in a virtual environment, which is required in order to support duplicating handles to the child or stealing handles from the parent. And the startup code in the child process is made aware of virtual environment via the "__PYVENV_LAUNCHER__" environment variable.

New changeset 0be3246 by Steve Dower (Adam Meily) in branch 'master':
bpo-39439: Fix multiprocessing spawn path in a venv on Windows (GH-18158)
0be3246

New changeset dbb37aa by Miss Islington (bot) in branch '3.8':
bpo-39439: Fix multiprocessing spawn path in a venv on Windows (GH-18158)
dbb37aa