Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow venv to create copies, even when symlinks are supported #63007

Closed
andreacorbellini mannequin opened this issue Aug 22, 2013 · 9 comments
Closed

Allow venv to create copies, even when symlinks are supported #63007

andreacorbellini mannequin opened this issue Aug 22, 2013 · 9 comments
Assignees
Labels
stdlib Python modules in the Lib dir

Comments

@andreacorbellini
Copy link
Mannequin

andreacorbellini mannequin commented Aug 22, 2013

BPO 18807
Nosy @warsaw, @vsajip
Files
  • venv-symlink-fix.patch: patch for Lib/venv/init.py
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/vsajip'
    closed_at = <Date 2013-08-24.23:04:36.831>
    created_at = <Date 2013-08-22.13:03:46.698>
    labels = ['library']
    title = 'Allow venv to create copies, even when symlinks are supported'
    updated_at = <Date 2014-06-04.17:55:57.490>
    user = 'https://bugs.python.org/andreacorbellini'

    bugs.python.org fields:

    activity = <Date 2014-06-04.17:55:57.490>
    actor = 'barry'
    assignee = 'vinay.sajip'
    closed = True
    closed_date = <Date 2013-08-24.23:04:36.831>
    closer = 'python-dev'
    components = ['Library (Lib)']
    creation = <Date 2013-08-22.13:03:46.698>
    creator = 'andrea.corbellini'
    dependencies = []
    files = ['35382']
    hgrepos = []
    issue_num = 18807
    keywords = ['patch']
    message_count = 9.0
    messages = ['195883', '196104', '219234', '219236', '219241', '219242', '219257', '219258', '219765']
    nosy_count = 5.0
    nosy_names = ['barry', 'vinay.sajip', 'andrea.corbellini', 'python-dev', 'Dominic.Cerquetti']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue18807'
    versions = ['Python 3.4', 'Python 3.5']

    @andreacorbellini
    Copy link
    Mannequin Author

    andreacorbellini mannequin commented Aug 22, 2013

    I'd really appreciate if venv could create environments without symlinks.

    Working on many Python projects, each one with different requirements, I prefer to keep everything I need in a single virtualenv directory, rather than two (one for the virtualenv and one for the built Python).

    So I'd like to have a --copies option that lets me force venv not to create symlinks. I can work on a patch if this issue is accepted.

    @andreacorbellini andreacorbellini mannequin added the stdlib Python modules in the Lib dir label Aug 22, 2013
    @vsajip vsajip self-assigned this Aug 23, 2013
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Aug 24, 2013

    New changeset ffb01a6c0960 by Vinay Sajip in branch 'default':
    Closes bpo-18807: pyvenv now takes a --copies argument allowing copies instead of symlinks even where symlinks are available and the default.
    http://hg.python.org/cpython/rev/ffb01a6c0960

    @python-dev python-dev mannequin closed this as completed Aug 24, 2013
    @DominicCerquetti
    Copy link
    Mannequin

    DominicCerquetti mannequin commented May 27, 2014

    Requesting re-open of this issue, using --closes to force no symlinks to be created still results in venv trying to create symlinks.

    I'm using Python 3.4 with the following command inside a vagrant Ubuntu 14.04 virtualbox image. The folder is a SMB mount from a windows host, which does not allow symlinks.

    Expected behavior: os.symlink() is never called when you run:
    python3.4 -m venv --copies

    Actual behavior: os.symlink() is still called in a few places such as:
    http://hg.python.org/cpython/file/b8e4bb1e1090/Lib/venv/__init__.py
    line: 147
    line: 215

    I have a fix for line 215 that I'm testing now (basically just need to call copier() instead of os.symlink()).

    I don't want to mess with line 147 due to it being OSX specific and I have no way to test it. But in theory it should also just be a call to copier()

    @vsajip
    Copy link
    Member

    vsajip commented May 27, 2014

    While you may be right about line 215, line 147 isn't analogous, because in the latter case it's a symlink to a directory. We don't really want the entire lib directory tree *copied* into lib64 (and in any case, the populating of lib will happen much after venv creatuion, when things are installed into the venv: a copy is of no use here).

    @DominicCerquetti
    Copy link
    Mannequin

    DominicCerquetti mannequin commented May 27, 2014

    Preliminary patch for line 215, per earlier description.

    While this doesn't appear to break anything and creates both copies and symlinks correctly, I do have these four failing unit tests:

    test_multiprocessing_fork
    test_multiprocessing_forkserver
    test_multiprocessing_main_handling
    test_multiprocessing_spawn

    I don't have time right now to look into it to see if they're related to my change (at first glance, it looks like not).

    @DominicCerquetti
    Copy link
    Mannequin

    DominicCerquetti mannequin commented May 27, 2014

    Ok cool, as you said line 215 then seems to be the only one that needs it.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented May 28, 2014

    New changeset ce1b8b2ddf07 by Vinay Sajip in branch '3.4':
    Issue bpo-18807: If copying (no symlinks) specified for a venv, then the python interpreter aliases (python, python3) are now created by copying rather than symlinking.
    http://hg.python.org/cpython/rev/ce1b8b2ddf07

    New changeset f2adaccc13ab by Vinay Sajip in branch 'default':
    Issue bpo-18807: Merged fix from 3.4.
    http://hg.python.org/cpython/rev/f2adaccc13ab

    @vsajip
    Copy link
    Member

    vsajip commented May 28, 2014

    I've made the change - not exactly the same as your patch, which was missing an os.chmod() and doing an unnecessary os.path.join() - but thanks for submitting a patch :-).

    However, note that on 64-bit Linux systems (actually any POSIX other than OS X) a symlink lib64 -> lib is still created. Perhaps this could be omitted, but I'm not sure if that would cause problems with pip. I've posted a note on the relevant issue - bpo-21197.

    @warsaw
    Copy link
    Member

    warsaw commented Jun 4, 2014

    The os.chmod() will fail if path is a symlink. At the very least it must be guarded by a not os.path.islink() call like above it. I'll add this check to 3.4 and 3.5.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants