This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: `python3 -m venv NAME`: virtualenv is not portable
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: CameronHudson8, Epic_Wink, Marco Sulla, vinay.sajip
Priority: normal Keywords:

Created on 2019-05-19 12:07 by Marco Sulla, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (18)
msg342842 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2019-05-19 12:07
I'm telling about python3 -m venv VIRTUALENV_NAME, not about the virtualenv binary.

Some remarks:

1. `VIRTUAL_ENV` variable in `activate` script is the absolute path of the virtualenv folder

2. A symlink to the `python3` bin of the machine is created.

This makes the virtualenv difficult to export to another machine. The VIRTUAL_ENV variable must be manually changed. 

Furthermore I do not understand why the simlink is created. I suppose that `python3` is already on the `PATH`, so what's the purpose of simlink?

I propose to makes VIRTUAL_ENV eqauls to the parent folder of the directory where `activate` resides. It makes it possible to move the virtualenv and copy it to another machine with the same OS.
msg342891 - (view) Author: Laurie Opperman (Epic_Wink) * Date: 2019-05-20 08:10
> Furthermore I do not understand why the simlink is created. I suppose that `python3` is already on the `PATH`, so what's the purpose of simlink?

On machines with multiple Python installs (eg Python 3.6 and Python 3.7, or a distributed Python 3.7 and a user-built Python 3.7), `python3` from PATH may refer to the incorrect installed version. Having a symlink to the the Python executable which built the environment (by default; `--copies` overcomes this) forces an executable.

This `python3` symlink still makes it distributable to other machines on the same platform anyway: `/usr/bin/python3` should always be available on other distributions with Python installed normally, and Windows copies the Python executable anyway AFAIK.

Across-platform is likely not going to work anyway: it's likely that your virtual environment will contain platform-specific installations of the packages, meaning they won't work on other platforms. If you just want Linux/MacOS portability, check out the Pex project.

I agree with VIRTUAL_ENV being relative however
msg342954 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2019-05-20 21:01
Well, I didn't know `--copy`. I think I'll use it. :)

What about VIRTUAL_ENV="$(dirname "$(dirname "$(readlink -nf "$0")")")"? In `bash` and in `sh` it works.
msg343326 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2019-05-23 20:52
Virtual environments are not designed to be portable. For instance, if you have entry points installed then moving them to another machine would break their shebang lines. And even if you do it on your local machine there's no guarantee something else wasn't structured to be directory-specific.

So thanks for opening the issue, but I'm closing as not a bug.
msg343519 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2019-05-25 22:12
> if you have entry points installed then moving them to another 
> machine would break their shebang lines.

Not if you port it on the same OS using, for example

#!/usr/bin/env python3

> And even if you do it on your local machine there's no guarantee
> something else wasn't structured to be directory-specific.

You are telling about user code, not the virtualenv itself. If the user doe not write the code in such a way it's portable, it's his fault. But this should not stop people that do it right to try to port the venv if possible.

I think the modification
VIRTUAL_ENV="$(dirname "$(dirname "$(readlink -nf "$0")")")"

is very little and quite robust. Don't know how to do in fish and csh shells, but in bash and sh it works.
msg344285 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2019-06-02 13:38
Well, what about the modification to VIRTUAL_ENV? I think it's little and without harm.
msg344468 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2019-06-03 19:27
Changing VIRTUAL_ENV will break code, so I'm not sure if it's worth changing.
msg344523 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2019-06-04 04:29
> Changing VIRTUAL_ENV will break code

VIRTUAL_ENV it's the same if you don't move the venv. Moving it will be an unofficial unsupported bonus, and if you coded bad and it doesn't work, it's your fault.
msg344572 - (view) Author: Laurie Opperman (Epic_Wink) * Date: 2019-06-04 13:41
If you're moving the virtual environment directory in Linux you'll also need to update the shebangs for the scripts/binaries
msg344624 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2019-06-04 19:02
"if you coded bad and it doesn't work, it's your fault."

I disagree. Your request changes what VIRTUAL_ENV gets set to, correct? Changing it suddenly for users doesn't mean their code was bad, it means we changed something on them. Now we can do that with a good justification, but simply to make it easier to have "unofficial unsupported bonus" is not a good enough reason to upset people with requiring everyone to update their code.
msg344626 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2019-06-04 19:04
Well, I repeat, not with 

#!/usr/bin/env python3
msg344629 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2019-06-04 19:07
The previous post was for Laurie Opperman

"upset people with requiring everyone to update their code"

I don't know why they have to be upset. Until now they can't move the folder. They want to move the folder? They have to change their code. Now they have no possibility at all to move it, so I don't know why they should be upset. 

And yes, I consider a code that depends from the path of the folder it resides a bad code.
msg344632 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2019-06-04 19:12
Furthermore, if you destroy an old virtual env and recreate it with the new method, it continues to work as before, since VIRTUAL_ENV points to the same folder. We don't force to change the code if they continues to use the virtual environments as now.
msg345004 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2019-06-07 19:56
Please Mr. Cannon, can you read my last posts? I think they are not describing a mad idea, but something reasonable.
msg345010 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2019-06-07 21:00
Please do not add me back to the nosy list as I stand by my thinking that this feature isn't worth pursuing. I understand you feel they are reasonable, Marco, but I don't like the idea of changing what VIRTUAL_ENV gets set to when I believe you should recreate the virtual environment as necessary and risk surprising people who expect VIRTUAL_ENV to function as it does today and has for years.
msg345756 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2019-06-16 14:15
> I don't like the idea of changing what VIRTUAL_ENV gets set to when I 
> believe you should recreate the virtual environment as necessary and 
> risk surprising people who expect VIRTUAL_ENV to function as it does 
> today and has for years.

(Talking to the wind) Well, didn't you have software tests?
msg382334 - (view) Author: Cameron Hudson (CameronHudson8) Date: 2020-12-02 20:13
@MarcoSulla thank you for opening this issue, it is a very necessary improvement.

My use case: I install packages one machine inside VPN, which downloads from a private pypi repo. I then package this as a zip file and deploy it on a cloud-hosted VM. I cannot simply reinstall the dependencies on the VM, because it can't reach the private pypi repo.

I have not contributed to Python before, but I am happy to help with reviewing a PR with the changes that you suggested.
msg382346 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2020-12-02 21:59
The PR will probably be rejected... you can do something like this:

1. in the venv on our machine, do `pip freeze`. This gives you the whole list of installed dependencies
2. download all the packages using `pip download`
3. copy all the packages on the cloud, create the venv and install them using `pip install $PATH_TO_PACKAGE`
History
Date User Action Args
2022-04-11 14:59:15adminsetgithub: 81145
2020-12-02 21:59:40Marco Sullasetmessages: + msg382346
2020-12-02 20:13:46CameronHudson8setnosy: + CameronHudson8
messages: + msg382334
2019-06-16 14:15:29Marco Sullasetmessages: + msg345756
2019-06-07 21:00:37brett.cannonsetnosy: - brett.cannon
2019-06-07 21:00:27brett.cannonsetnosy: brett.cannon, vinay.sajip, Epic_Wink, Marco Sulla
messages: + msg345010
2019-06-07 19:56:12Marco Sullasetnosy: + brett.cannon
messages: + msg345004
2019-06-04 19:12:15Marco Sullasetmessages: + msg344632
2019-06-04 19:10:41brett.cannonsetnosy: - brett.cannon
2019-06-04 19:07:59Marco Sullasetmessages: + msg344629
2019-06-04 19:04:11Marco Sullasetmessages: + msg344626
2019-06-04 19:02:22brett.cannonsetmessages: + msg344624
2019-06-04 13:41:33Epic_Winksetmessages: + msg344572
2019-06-04 04:29:06Marco Sullasetmessages: + msg344523
2019-06-03 19:27:08brett.cannonsetmessages: + msg344468
2019-06-02 13:38:01Marco Sullasetmessages: + msg344285
2019-05-25 22:12:15Marco Sullasetmessages: + msg343519
2019-05-23 20:52:06brett.cannonsetstatus: open -> closed

nosy: + brett.cannon
messages: + msg343326

resolution: not a bug
stage: resolved
2019-05-20 21:01:05Marco Sullasetmessages: + msg342954
2019-05-20 08:10:19Epic_Winksetnosy: + Epic_Wink
messages: + msg342891
2019-05-19 14:27:04SilentGhostsetnosy: + vinay.sajip

versions: + Python 3.8, - Python 3.9
2019-05-19 12:07:19Marco Sullacreate