Issue32444
Created on 2017-12-29 02:48 by seliger, last changed 2020-04-27 07:37 by ned.deily. This issue is now closed.
Messages (5) | |||
---|---|---|---|
msg309157 - (view) | Author: Corey Seliger (seliger) | Date: 2017-12-29 02:48 | |
The venv module does not evenly create the necessary binaries/symlinks when building virtualenvs. If you call venv like this: seliger@core:~/venvs$ python3 -m venv venvA seliger@core:~/venvs$ ls -l venvA/bin/ total 32 -rw-r--r-- 1 seliger seliger 2143 Dec 28 21:29 activate -rw-r--r-- 1 seliger seliger 1259 Dec 28 21:29 activate.csh -rw-r--r-- 1 seliger seliger 2397 Dec 28 21:29 activate.fish -rwxrwxr-x 1 seliger seliger 254 Dec 28 21:29 easy_install -rwxrwxr-x 1 seliger seliger 254 Dec 28 21:29 easy_install-3.5 -rwxrwxr-x 1 seliger seliger 226 Dec 28 21:29 pip -rwxrwxr-x 1 seliger seliger 226 Dec 28 21:29 pip3 -rwxrwxr-x 1 seliger seliger 226 Dec 28 21:29 pip3.5 lrwxrwxrwx 1 seliger seliger 7 Dec 28 21:29 python -> python3 lrwxrwxrwx 1 seliger seliger 16 Dec 28 21:29 python3 -> /usr/bin/python3 ...you do not end up with a "python3.5" binary. However, if you call venv like this: seliger@core:~/venvs$ python3.5 -m venv venvB seliger@core:~/venvs$ ls -l venvB/bin total 32 -rw-r--r-- 1 seliger seliger 2143 Dec 28 21:29 activate -rw-r--r-- 1 seliger seliger 1259 Dec 28 21:29 activate.csh -rw-r--r-- 1 seliger seliger 2397 Dec 28 21:29 activate.fish -rwxrwxr-x 1 seliger seliger 256 Dec 28 21:29 easy_install -rwxrwxr-x 1 seliger seliger 256 Dec 28 21:29 easy_install-3.5 -rwxrwxr-x 1 seliger seliger 228 Dec 28 21:29 pip -rwxrwxr-x 1 seliger seliger 228 Dec 28 21:29 pip3 -rwxrwxr-x 1 seliger seliger 228 Dec 28 21:29 pip3.5 lrwxrwxrwx 1 seliger seliger 9 Dec 28 21:29 python -> python3.5 lrwxrwxrwx 1 seliger seliger 9 Dec 28 21:29 python3 -> python3.5 lrwxrwxrwx 1 seliger seliger 18 Dec 28 21:29 python3.5 -> /usr/bin/python3.5 ...you DO get the necessary python3.5 binary. Some vendors are making it a requirement to call a specific pythonX.Y binary to ensure compatibility. One such example is the serverless framework when deploying to Amazon Web Services. Another example is the useful pyenv utility that manages full Python builds and virtualenvs. When it depends upon venv, it exhibits the same behavior. I submitted a patch workaround to force calling venv using pythonX.Y, but this really seems like an issue with the venv module itself. The expected behavior should be that venv generates all three binaries (python, python3, and python3.5) regardless of how the python command was invoked. I am able to reproduce this on Python 3.5 and 3.6. I could not find any other similar references in searching the bug system. |
|||
msg309158 - (view) | Author: Corey Seliger (seliger) | Date: 2017-12-29 02:54 | |
Here is the write-up I posted over on the pyenv issues page: This is apparently a behavior issue in the way venv works. Not only does this impact 3.6.x, I also confirmed that the same issue occurs with venv in general using the stock 3.5 Python that comes with Ubuntu 16.04. The trouble is at Lib/venv/__init__.py:111. Starting at line 111: if sys.platform == 'darwin' and '__PYVENV_LAUNCHER__' in env: executable = os.environ['__PYVENV_LAUNCHER__'] else: executable = sys.executable dirname, exename = os.path.split(os.path.abspath(executable)) context.executable = executable This carries down to the setup_python method starting at line 187 (Lib/venv/__init__.py:187). It takes whatever executable that is called as the source, and then creates the others based on a hard coded list (e.g. python and python3). def setup_python(self, context): """ Set up a Python executable in the environment. :param context: The information for the environment creation request being processed. """ binpath = context.bin_path path = context.env_exe copier = self.symlink_or_copy copier(context.executable, path) dirname = context.python_dir if os.name != 'nt': if not os.path.islink(path): os.chmod(path, 0o755) for suffix in ('python', 'python3'): path = os.path.join(binpath, suffix) if not os.path.exists(path): # Issue 18807: make copies if # symlinks are not wanted copier(context.env_exe, path, relative_symlinks_ok=True) if not os.path.islink(path): os.chmod(path, 0o755) else: subdir = 'DLLs' include = self.include_binary files = [f for f in os.listdir(dirname) if include(f)] for f in files: src = os.path.join(dirname, f) dst = os.path.join(binpath, f) if dst != context.env_exe: # already done, above copier(src, dst) dirname = os.path.join(dirname, subdir) if os.path.isdir(dirname): files = [f for f in os.listdir(dirname) if include(f)] for f in files: src = os.path.join(dirname, f) dst = os.path.join(binpath, f) copier(src, dst) # copy init.tcl over for root, dirs, files in os.walk(context.python_dir): if 'init.tcl' in files: tcldir = os.path.basename(root) tcldir = os.path.join(context.env_dir, 'Lib', tcldir) if not os.path.exists(tcldir): os.makedirs(tcldir) src = os.path.join(root, 'init.tcl') dst = os.path.join(tcldir, 'init.tcl') shutil.copyfile(src, dst) break If you don't call -m venv as python3.X, you will never get that binary in the bin directory. |
|||
msg309173 - (view) | Author: R. David Murray (r.david.murray) * ![]() |
Date: 2017-12-29 14:09 | |
This is a feature that actually supports your use case, as well as the use cases of those who *don't* want to strap the python version: you get what you ask for. If you call venv with 'python3', you get a venv that will use your new python if you upgrade your python, but if you call it with python3.x, you will get a venv that will *not* upgrade, which is exactly what you want for your use case. |
|||
msg309174 - (view) | Author: R. David Murray (r.david.murray) * ![]() |
Date: 2017-12-29 14:15 | |
Actually, I'm going to reopen this as a doc issue because this behavior is not discussed by the docs that I can see, and it is important to know about when creating a venv. |
|||
msg367406 - (view) | Author: Ned Deily (ned.deily) * ![]() |
Date: 2020-04-27 07:37 | |
The behavior of venv has been changed in Python 3.9 to always create a "pythonM.N" link in the venv bin directory, in addition to "pythonM", regardless of how venv was invoked. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2020-04-27 07:37:57 | ned.deily | set | status: open -> closed superseder: shebanged scripts can escape from `venv` depending on how it was created versions: + Python 3.9, - Python 3.6, Python 3.7 nosy: + ned.deily messages: + msg367406 resolution: duplicate stage: needs patch -> resolved |
2017-12-29 14:15:09 | r.david.murray | set | status: closed -> open assignee: docs@python components: + Documentation, - Library (Lib) title: python -m venv has incongruous behavor creating binaries -> python -m venv symlink dependency on how python binary is called is not documented nosy: + docs@python messages: + msg309174 resolution: not a bug -> (no value) stage: resolved -> needs patch |
2017-12-29 14:09:49 | r.david.murray | set | status: open -> closed versions: + Python 3.7, - Python 3.5 nosy: + r.david.murray messages: + msg309173 resolution: not a bug stage: resolved |
2017-12-29 02:54:22 | seliger | set | messages: + msg309158 |
2017-12-29 02:48:17 | seliger | create |