The solution to work around the bug is there:

http://ipoveraviancarriers.blogspot.it/2012/11/python-33-and-pyvenv-hackish-solution.html

But probably pyvenv should be patched in order to create symlinks.

Thanks!

What problem are you trying to solve here?  pyvenv is working as documented.  And, although pyvenv does not claim to be compatible with the third-party virtualenv package, as far as I know virtualenv does not create links for local/bin and friends either.

> as far as I know virtualenv does not create links for local/bin and friends either.

AFAIK it never used to, but it seems to do so since 1.6.3, in an attempt to fix a problem with the "posix_local" install scheme. I just tested with virtualenv 1.6.4, and when I create a venv in /tmp/venv, it creates a symbolic link /tmp/venv/local which points back to /tmp/venv. This behaviour causes problems with e.g. PyCharm, which shows ever-nesting directories ("turtles all the way down"), has led to later versions of virtualenv creating an actual directory and symlinking the venv's contents to entries in it.

The question is: is this still necessary? Why exactly does "posix_local" need this, and does it need it for Python 3.3, 3.4 etc? Adding Carl to nosy, hoping he can shed some light on this.

Whoops, mouse glitch led to components being wrongly set.

> The question is: is this still necessary? Why exactly does
> "posix_local" need this, and does it need it for Python 3.3, 3.4 etc? 
> Adding Carl to nosy, hoping he can shed some light on this.

I hope to see the light on that too. ;-)

Meanwhile, my problem was that after creating the virtualenv, as cited in the blog post above linked, I got stuck in installing pip (and thus distribute) in that virtualenv, because 'VENV/local/bin' is not in the $PATH (and it seems reasonable for a virtualenv that this will remain stable).

So, (IMHO) if changing the $PATH is not a good idea, either python should do:

1. patch distribute and pip to do not install in ./local by default if in a VENV. (enviroment variable based?, whatever)

2. create the links after creating the VENV (my naïve solution)

Am I right or I completely hit the wrong issue?

Here is the bug filed against virtualenv that led to the addition of the local/ directory: https://github.com/pypa/virtualenv/issues/118

As Vinay pointed out, the original fix was later modified to be friendlier to tools that dislike recursive symlinks.

That's about all I know; I don't know if any of this is still needed in 3.3/3.4.

2012/11/21 Carl Meyer <report@bugs.python.org>

>
> Carl Meyer added the comment:
>
> Here is the bug filed against virtualenv that led to the addition of the
> local/ directory: https://github.com/pypa/virtualenv/issues/118
>
> As Vinay pointed out, the original fix was later modified to be friendlier
> to tools that dislike recursive symlinks.
>

I didn't used virtualenv but pyvenv-3.3, so if this is not a bug, my
question become:

how to proper install distribute and pip in python 3.3 venv since they
installs in <VENV>/local/bin and that <VENV>/bin/activate does not
include <VENV>/local/bin in $PATH ?

What OS are you on, Marco?

It looks to me like pyvenv probably does need the same hack as virtualenv here, to deal with OSes who set posix_local as the default installation scheme.

On Wednesday 21 November 2012 21:32:57 Carl Meyer wrote:
> Carl Meyer added the comment:
> 
> What OS are you on, Marco?

Linux. Debian GNU/Linux amd64 sid/experimental.

> It looks to me like pyvenv probably does need the same hack as virtualenv
> here, to deal with OSes who set posix_local as the default installation
> scheme.

nice!

Is it impossible to edit the install scheme when run in a venv, so that the workaround becomes unnecessary?  We are the upstream after all :)

IIUC, Debian Experimental is not a complete distribution. Are there other, non-experimental OS variants where posix_local is used with Python 3.x?

On my Ubuntu Oneiric and Precise 64-bit machines, the default install scheme on the official Python 3.2 - as determined by sysconfig._get_default_scheme() - is "posix_prefix".

having the posix_prefix as the default in Debian is an oversight on my side. it always should be posix_local. I'll fix this at least for 3.3 in current development releases.

The rationale for this is that distutils based installs install by default into the path which is used by the distributor and get in conflict with the packages distributed by Debian/Ubuntu.

At one of the language summits in Chicago I asked that the default installation target should be changed to the user dir, but this idea was rejected.

Therefore you do have two site directories, called

  /usr/lib/pythonX.Y/dist-packages
  /usr/local/lib/pythonX.Y/dist-packages

The renaming from site to dist was done to not conflict with a default python installation (without any configure parameters).

The default install should go to /usr/local, packagers for Debian usually should use the Debian local distutils option --install-layout=deb.

Even when "posix_local" becomes the default scheme on Debian python builds, it's not clear whether the suggested fix is the correct one to make, unless "posix_local" is added to sysconfig's list of _INSTALL_SCHEMES. Matthias comment makes sense when considering system-wide installs, but when installing into a venv, these considerations don't seem to apply.

Technically the change is not hard to do, but I'm a little uncomfortable whether it's the right thing to do, as it's only for a specific Linux distribution, and it seems a bit of a kludge.

Although the ideal situation is for software not to care whether it's running in a venv, I'm not sure whether installers like pip/distribute need to be completely ignorant of whether they're installing in a venv.

For example, the posix_local scheme says that the site-packages directory is to be called 'dist-packages', which seems unnecessary in a venv since its purpose is really for system-wide installations. Yet, virtualenv names the directory 'site-packages', so it appears not to be following the scheme faithfully as far as that part of it is concerned.

virtualenv for 2.7, when used with a Debian/Ubuntu system installation, should only see site-packages.  Same thing should be done for venv for 3.3 and up. Even if needed patches will not go upstream, they are welcome.

fixed in experimental, pyvenv's are now handled the same way as python-virtualenvs. closing the issue here.

Could you please tell me, in which exact release or commit this is resolved?

I'm using Python 3.3.0 on Ubuntu Precise and I noticed like the others, that e.g. easy_install and pip are installed to venv/local/bin instead of venv/bin and the site-packages folder being used is venv/local/lib/python3.3/dist-packages instead of venv/lib/python3.3/site-packages.

Are both issues solved in experimental? So far I only saw a symlink-patch for virtualenv but that doesn't actually fix the problem of being double-local (venv and local-folder within venv), neither dist-packages:

https://github.com/pypa/virtualenv/commit/285679cfd326c918676e765e06ed142db66efde0

From Python’s viewpoint this is closed, please follow up on the Debian bug tracker.

Which bug ID? Couldn't find it anywhere in the previous comments.

It was linked from the blog post (see first message): https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=695758