Author pierreglaser
Recipients paul.moore, pierreglaser, pitrou, steve.dower, tim.golden, zach.ware
Date 2019-09-10.13:32:39
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1568122359.92.0.800539114246.issue38092@roundup.psfhosted.org>
In-reply-to
Content
If I am not mistaken, when creating a new process on Python3.7 and later on Windows, if using a virtualenv, Python now uses a launcher. The launcher is being notified that it must create a virtual-environment Python (and not a system Python) program using the __PYVENV_LAUNCHER__ environment variable, passed as part of the environment of launcher process created using in _winapi.CreateProcess
(see https://github.com/python/cpython/blob/9008be303a89bfab8c3314c6a42330b5523adc8b/Lib/multiprocessing/popen_spawn_win32.py#L73-L75)

However, if I am not mistaken `env` is not passed at the right position (https://github.com/python/cpython/blob/9008be303a89bfab8c3314c6a42330b5523adc8b/Modules/_winapi.c#L1062-L1068). These lines were part of a bugfix patch (see https://bugs.python.org/issue35797), solving an issue for multiprocessing-based packages. We ended trying to backport to loky (https://github.com/tomMoral/loky, a multiprocessing-based package) but the bug was not fixed. Passing 'env' correctly fixed the bug.

Two things:
- It is hard to provide a reproducer for this issue as it requires patching the CPython source code.
- I don't understand why env being not passed correctly does not manifest itself in the test-suite. What is even more troubling is that even with this bug, the virtualenv launcher seems to get that a virtualenv is used when creating processes in multiprocessing (at least, sys.path includes the path/to/venv/lib/python3.x/site-packages). However, I am not familiar with the virtualenv launcher for python3.7+ and windows.
History
Date User Action Args
2019-09-10 13:32:39pierreglasersetrecipients: + pierreglaser, paul.moore, pitrou, tim.golden, zach.ware, steve.dower
2019-09-10 13:32:39pierreglasersetmessageid: <1568122359.92.0.800539114246.issue38092@roundup.psfhosted.org>
2019-09-10 13:32:39pierreglaserlinkissue38092 messages
2019-09-10 13:32:39pierreglasercreate