classification
Title: Default preference not given to venv DLL's
Type: behavior Stage: patch review
Components: Library (Lib), Windows Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, jonathan-lp, paul.moore, steve.dower, tim.golden, vinay.sajip, zach.ware
Priority: normal Keywords: patch

Created on 2018-06-30 10:07 by jonathan-lp, last changed 2018-07-20 16:07 by vinay.sajip.

Pull Requests
URL Status Linked Edit
PR 8253 merged vinay.sajip, 2018-07-11 14:53
Messages (5)
msg320765 - (view) Author: Jonathan (jonathan-lp) Date: 2018-06-30 10:07
I don't know if this is a bug or an odd design decision or just something that hasn't been considered or maybe I'm missing something.

On Windows and I create a venv with Python 3.6.3:

> python -m venv venv

This creates a subdirectory called /venv. Inside this, there's:
./venv/Scripts/sqlite3.dll

This is the sqlite library - except it's not, because Python isn't using this file. If I upgrade this library by replacing it with a newer sqlite3.dll version, Python keep using the original version of the library. Turns out, Python is by default reading the DLL in the root Python install:

c:\Python36\DLLs\sqlite3.dll

Now, I can change that file and sure enough my VENV (*all* VENVs) then get the newer version of SQLite, or I can delete that file and the VENV's will all use their local versions, or I can possibly play with some sys.path to try and get the VENV version loaded first.

But this raises a few questions:
1) This seems like a rather odd default - copying a file that is never used by default.
2) Surely either the correct option is to use the VENV version by default?
3) Otherwise, what's the point in copying across this DLL file into the VENV by default?
msg320932 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2018-07-03 01:56
Offhand I don't know why it copies PYD and DLL files from the DLLs directory. It's part of the standard library. The virtual environment should only need to copy or symlink the binaries in the application directory, such as python.exe, pythonw.exe, vcruntime<###>.dll, python<##>.dll, and python<#>.dll. Currently a virtual environment doesn't use the binaries that get copied or symlinked from the DLLs directory, since the system DLLs directory is in sys.path.

Also, why does it copy over init.tcl? _tkinter uses Py_GetPrefix() to to get the real prefix directory in order to find the TCL library at "tcl\tclX.y". For a Python 3.7 virtual environment, I verified using Sysinternals procmon that TCL loads the init file from "C:\Program Files\Python37\tcl\tcl8.6\init.tcl", not the virtual environment.
msg320934 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2018-07-03 05:20
The code that does the copy is the original PEP 405 implementation code (from 26 May 2012). I'm fairly sure that these files were copied over only because (at least prior to the Python 3.3 release in September 2012) they were needed to be in the venv for the venv to work correctly. Python internals may have changed subsequently, making these copies unnecessary.

I can't easily recreate a Python 3.3.0 on Windows to confirm this, but I'm pretty sure I only coded it to copy over what was actually needed at the time.
msg320935 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2018-07-03 05:54
Hmmm. I managed to find a machine to install Python3.3.0 on, and it appears that even with that copy suite removed, the tests still work. So possibly it's something that changed between 3.3.0 alpha and 3.3.0 final.

I can try removing the copy code for 3.8 but I'm not sure if any code out there is relying on the copied files being in the venv, so it may not be a good idea to backport.
msg322024 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2018-07-20 16:07
New changeset 94487d45707772723ef19e86700a40a12743baa1 by Vinay Sajip in branch 'master':
bpo-34011: Update code copying DLLs and init.tcl into venvs. (GH-8253)
https://github.com/python/cpython/commit/94487d45707772723ef19e86700a40a12743baa1
History
Date User Action Args
2018-07-20 16:07:40vinay.sajipsetmessages: + msg322024
2018-07-11 14:53:51vinay.sajipsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request7785
2018-07-03 05:54:07vinay.sajipsetmessages: + msg320935
2018-07-03 05:20:46vinay.sajipsetmessages: + msg320934
2018-07-03 01:58:12eryksunsetversions: + Python 3.7, Python 3.8
2018-07-03 01:56:39eryksunsetnosy: + eryksun, paul.moore, tim.golden, vinay.sajip, zach.ware, steve.dower
messages: + msg320932

components: + Library (Lib), Windows
type: behavior
stage: needs patch
2018-06-30 10:07:43jonathan-lpcreate