classification
Title: Allow venv to create copies, even when symlinks are supported
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: vinay.sajip Nosy List: Dominic.Cerquetti, andrea.corbellini, barry, python-dev, vinay.sajip
Priority: normal Keywords: patch

Created on 2013-08-22 13:03 by andrea.corbellini, last changed 2014-06-04 17:55 by barry. This issue is now closed.

Files
File name Uploaded Description Edit
venv-symlink-fix.patch Dominic.Cerquetti, 2014-05-27 21:25 patch for Lib/venv/__init__.py review
Messages (9)
msg195883 - (view) Author: Andrea Corbellini (andrea.corbellini) Date: 2013-08-22 13:03
I'd really appreciate if `venv` could create environments without symlinks.

Working on many Python projects, each one with different requirements, I prefer to keep everything I need in a single virtualenv directory, rather than two (one for the virtualenv and one for the built Python).

So I'd like to have a --copies option that lets me force venv not to create symlinks. I can work on a patch if this issue is accepted.
msg196104 - (view) Author: Roundup Robot (python-dev) Date: 2013-08-24 23:04
New changeset ffb01a6c0960 by Vinay Sajip in branch 'default':
Closes #18807: pyvenv now takes a --copies argument allowing copies instead of symlinks even where symlinks are available and the default.
http://hg.python.org/cpython/rev/ffb01a6c0960
msg219234 - (view) Author: Dominic Cerquetti (Dominic.Cerquetti) * Date: 2014-05-27 18:56
Requesting re-open of this issue, using --closes to force no symlinks to be created still results in venv trying to create symlinks.

I'm using Python 3.4 with the following command inside a vagrant Ubuntu 14.04 virtualbox image.  The folder is a SMB mount from a windows host, which does not allow symlinks.

Expected behavior: os.symlink() is never called when you run:
python3.4 -m venv --copies

Actual behavior: os.symlink() is still called in a few places such as:
http://hg.python.org/cpython/file/b8e4bb1e1090/Lib/venv/__init__.py
line: 147
line: 215

I have a fix for line 215 that I'm testing now (basically just need to call copier() instead of os.symlink()). 

I don't want to mess with line 147 due to it being OSX specific and I have no way to test it.  But in theory it should also just be a call to copier()
msg219236 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2014-05-27 20:04
While you may be right about line 215, line 147 isn't analogous, because in the latter case it's a symlink to a directory. We don't really want the entire lib directory tree *copied* into lib64 (and in any case, the populating of lib will happen much after venv creatuion, when things are installed into the venv: a copy is of no use here).
msg219241 - (view) Author: Dominic Cerquetti (Dominic.Cerquetti) * Date: 2014-05-27 21:25
Preliminary patch for line 215, per earlier description.

While this doesn't appear to break anything and creates both copies and symlinks correctly, I do have these four failing unit tests:

test_multiprocessing_fork 
test_multiprocessing_forkserver
test_multiprocessing_main_handling 
test_multiprocessing_spawn

I don't have time right now to look into it to see if they're related to my change (at first glance, it looks like not).
msg219242 - (view) Author: Dominic Cerquetti (Dominic.Cerquetti) * Date: 2014-05-27 21:33
Ok cool, as you said line 215 then seems to be the only one that needs it.
msg219257 - (view) Author: Roundup Robot (python-dev) Date: 2014-05-28 07:08
New changeset ce1b8b2ddf07 by Vinay Sajip in branch '3.4':
Issue #18807: If copying (no symlinks) specified for a venv, then the python interpreter aliases (python, python3) are now created by copying rather than symlinking.
http://hg.python.org/cpython/rev/ce1b8b2ddf07

New changeset f2adaccc13ab by Vinay Sajip in branch 'default':
Issue #18807: Merged fix from 3.4.
http://hg.python.org/cpython/rev/f2adaccc13ab
msg219258 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2014-05-28 07:16
I've made the change - not exactly the same as your patch, which was missing an os.chmod() and doing an unnecessary os.path.join() - but thanks for submitting a patch :-).

However, note that on 64-bit Linux systems (actually any POSIX other than OS X) a symlink lib64 -> lib is still created. Perhaps this could be omitted, but I'm not sure if that would cause problems with pip. I've posted a note on the relevant issue - #21197.
msg219765 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2014-06-04 17:55
The os.chmod() will fail if path is a symlink.  At the very least it must be guarded by a `not os.path.islink()` call like above it.  I'll add this check to 3.4 and 3.5.
History
Date User Action Args
2014-06-04 17:55:57barrysetnosy: + barry
messages: + msg219765
2014-05-28 07:16:13vinay.sajipsetmessages: + msg219258
2014-05-28 07:08:47python-devsetmessages: + msg219257
2014-05-27 21:33:43Dominic.Cerquettisetmessages: + msg219242
2014-05-27 21:25:30Dominic.Cerquettisetfiles: + venv-symlink-fix.patch
keywords: + patch
messages: + msg219241
2014-05-27 20:04:02vinay.sajipsetmessages: + msg219236
2014-05-27 18:56:36Dominic.Cerquettisetnosy: + Dominic.Cerquetti
messages: + msg219234
2013-08-24 23:04:36python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg196104

resolution: fixed
stage: resolved
2013-08-23 11:20:52vinay.sajipsetassignee: vinay.sajip
versions: + Python 3.4
2013-08-23 08:22:12pitrousetnosy: + vinay.sajip
2013-08-22 13:03:46andrea.corbellinicreate