classification
Title: Support for relative home path in pyvenv.cfg
Type: enhancement Stage: patch review
Components: Interpreter Core Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Jeff.Edwards, brett.cannon, eryksun, steve.dower, vinay.sajip
Priority: normal Keywords: patch

Created on 2020-01-27 20:58 by Jeff.Edwards, last changed 2020-02-05 00:08 by steve.dower.

Pull Requests
URL Status Linked Edit
PR 18213 open python-dev, 2020-01-27 21:00
Messages (6)
msg360800 - (view) Author: Jeff Edwards (Jeff.Edwards) * Date: 2020-01-27 20:58
Currently, the interpreter only supports absolute paths for the 'home' directory in the pyvenv.cfg file.  While this works when the interpreter is always installed at a fixed location, it impacts the portability of virtual environments and can make it notably more-difficult if multiple virtual environments are shipped with a shared interpreter and are intended to be portable and working in any directory.

Many of these issues can be solved for if 'home' can use a directory relative to the directory of the pyvenv.cfg file.  This is detected by the presence of a starting '.' in the value.

A common use-case for this is that a script-based tool (e.g. black or supervisor) may be shipped with a larger portable application where they are intended to share the same interpreter (to save on deployment size), but may have conflicting dependencies.  Since the application only depends on the executable scripts, those packages could be packaged into  their own virtual environments with their dependencies.
msg360886 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-01-28 18:27
Do note that virtual environments are not designed to be portable in general, so this would be a fundamental change in the design and purpose of virtual environments.
msg360887 - (view) Author: Jeff Edwards (Jeff.Edwards) * Date: 2020-01-28 18:32
I would say they’re not designed to be, but the also aren’t designed to not
be portable.  This is often useful where open network access isn’t
reasonable, so access to Pip/pipx/pipenv is limited at best.

Suffice to say, is there a significant reason to not allow it?

On Tue, Jan 28, 2020 at 10:28 AM Brett Cannon <report@bugs.python.org>
wrote:

>
> Brett Cannon <brett@python.org> added the comment:
>
> Do note that virtual environments are not designed to be portable in
> general, so this would be a fundamental change in the design and purpose of
> virtual environments.
>
> ----------
> nosy: +brett.cannon, vinay.sajip
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue39469>
> _______________________________________
>
msg360893 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-01-28 18:39
> Suffice to say, is there a significant reason to not allow it?

We have to support that use-case forever. ;) In all seriousness, relative paths get tricky when you forget to resolve the path as appropriate (and in a way that people expect) and it requires making sure that this will work for the next 30 years, so it's not a small thing to take on.
msg360908 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2020-01-28 22:33
> Suffice to say, is there a significant reason to not allow it?

It's poorly supported by packaging. In particular, relocating an environment isn't supported with entry-point scripts, which pip installs with a fully-qualified shebang. Moreover, entry-point scripts in Windows are created as exe files (e.g. "pip.exe") that embed the fully-qualified path of python[w].exe in the environment, plus a zipped __main__.py. For example, given an environment at "C:\Temp\env", running "C:\Temp\env\Scripts\pip.exe" in turn spawns a child process with the command line: "C:\Temp\env\Scripts\python.exe" "C:\Temp\env\Scripts\pip.exe". This breaks if the environment is renamed or relocated.
msg360909 - (view) Author: Jeff Edwards (Jeff.Edwards) * Date: 2020-01-28 22:52
Interesting, I hadn’t realized that it would embed the FQ Executable path,
but that does make sense overall.  I guess I had always planned on fixing
the ‘bin’ directory anyway afterwards, it’s just that the lack of relative
home made it significantly harder to encapsulate multiple environments
running with the same interpreter without having to do a complete
reinstall, and venv did seem like the best and most-pythonic way to do it.

I’ll think about it a bit more

On Tue, Jan 28, 2020 at 2:33 PM Eryk Sun <report@bugs.python.org> wrote:

>
> Eryk Sun <eryksun@gmail.com> added the comment:
>
> > Suffice to say, is there a significant reason to not allow it?
>
> It's poorly supported by packaging. In particular, relocating an
> environment isn't supported with entry-point scripts, which pip installs
> with a fully-qualified shebang. Moreover, entry-point scripts in Windows
> are created as exe files (e.g. "pip.exe") that embed the fully-qualified
> path of python[w].exe in the environment, plus a zipped __main__.py. For
> example, given an environment at "C:\Temp\env", running
> "C:\Temp\env\Scripts\pip.exe" in turn spawns a child process with the
> command line: "C:\Temp\env\Scripts\python.exe"
> "C:\Temp\env\Scripts\pip.exe". This breaks if the environment is renamed or
> relocated.
>
> ----------
> nosy: +eryksun
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue39469>
> _______________________________________
>
History
Date User Action Args
2020-02-05 00:08:02steve.dowersetnosy: + steve.dower
2020-01-28 22:52:40Jeff.Edwardssetmessages: + msg360909
2020-01-28 22:33:48eryksunsetnosy: + eryksun
messages: + msg360908
2020-01-28 18:39:06brett.cannonsetmessages: + msg360893
2020-01-28 18:32:16Jeff.Edwardssetmessages: + msg360887
2020-01-28 18:27:56brett.cannonsetnosy: + vinay.sajip, brett.cannon
messages: + msg360886
2020-01-27 21:00:34python-devsetkeywords: + patch
stage: patch review
pull_requests: + pull_request17592
2020-01-27 20:58:08Jeff.Edwardscreate