This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Distutils2 ignores site-local configuration
Type: behavior Stage: resolved
Components: Distutils2 Versions: Python 3.3, 3rd party
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: eric.araujo Nosy List: alex.gronholm, alexis, barry, eric.araujo, tarek
Priority: normal Keywords:

Created on 2012-03-18 06:10 by alex.gronholm, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (6)
msg156230 - (view) Author: Alex Grönholm (alex.gronholm) * Date: 2012-03-18 06:10
Distutils2 seems to rely solely on a sysconfig.cfg shipped with distutils2 to get the path where to install packages.
Ignoring site-local configuration means that it won't work on Python distributions where the site configuration has been customized.

On Ubuntu, packages are supposed to go to /usr/local/lib/python2.7/dist-packages but distutils2 tries to install them to /usr/lib/python2.7/site-packages.
msg156247 - (view) Author: Alexis Metaireau (alexis) * (Python triager) Date: 2012-03-18 11:44
The supposed way to work, for OS packagers, is to ship this 
sysconfig.cfg thing.

I'm not sure we should rely on a customized site-local configuration, 
without defining any standard way of doing this (IOW: what are we 
looking for in the site-local config?)

Also, I would add that this seem to be a compatibility thing, maybe 
could we add a fallback to site-local if no sysconfig is found?
msg156265 - (view) Author: Alex Grönholm (alex.gronholm) * Date: 2012-03-18 18:50
>The supposed way to work, for OS packagers, is to ship this 
>sysconfig.cfg thing.

Even for Pythons older than 3.3?
msg156294 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-18 23:52
> Distutils2 seems to rely solely on a sysconfig.cfg shipped with distutils2 to get the path where to install packages.
That is correct.

> Ignoring site-local configuration
What do you mean?  sitecustomize is executed, for example.  If you only mean Debian/Ubuntu/derivatives-specific edits/kludges, read on :)

> On Ubuntu, packages are supposed to go to /usr/local/lib/python2.7/dist-packages but distutils2 tries to
> install them to /usr/lib/python2.7/site-packages.
Yes and no.

We need to make a difference between a Python installed with the Debuntu packaging tools (let’s call it the system Python) and a Python downloaded or cloned from python.org and installed locally by a sysadmin in /opt or /usr/local (let’s call it local Python).

In a local Python installed in /usr/local (location recommended over /opt by the Debian policy, if memory serves), third-party distributions get installed to /usr/local/lib/pythonX.Y/site-packages.  That’s good.

On Debian, the system Python used to look for third-party distributions in /usr/lib/pythonX.Y/site-packages.  One should never use tools like pip or pysetup to install things here, but only system packaging tools, i.e. dpkg and APT.  The recommended way to install things without using system tools, according to Debian’s interpretation of the File Hierarchy Standard, is to put them under /usr/local, so /usr/local/lib/pythonX.Y/site-packages.  (Just like /usr/local/bin completes what’s available in /usr/bin, /usr/local/lib/pythonX.Y completes /usr/lib/pythonX.Y.)  That’s good too.

If we combine these two sets of things we have a conflict: projects installed for use with a local Python may affect and break the system Python.  That happened to Barry, for example.  That’s why paths were changed in Debian and derivatives: now you can install things for a local Python in /usr/local/lib/pythonX.Y/site-packages and install other things for the system Python in /usr/local/lib/pythonX.Y/dist-packages.

Now, with Python 3.3 we have a change to make life easier for Debuntu: instead of patching the site and distutils modules, they can edit sysconfig.cfg and Python will automatically look for things in dist-packages and packaging will install to that directory too.  distutils will still need the manual patching.  A local Python will still use site-packages, or whatever the admin configures in sysconfig.cfg, and the system Python will use dist-packages, or whatever the Debian Python maintainer decides.

If there is a Debian package of distutils2 for older Pythons, then they’ll just have to patch d2/_backport/sysconfig.cfg.  The only corner cases that I could see are for example if an admin installs distutils2 for their system Python in /usr/local/lib/pythonX.Y/dist-packages: they will have to adapt d2/_backport/sysconfig.cfg, or use the setup.py script instead of pysetup (because setup.py uses distutils and thus will use the patched paths).

If I understand correctly that you used “/usr/bin/python pysetup install spam” and wanted it to install to /usr/lib/python2.7/site-packages, then I think that the correct reply is: Not supported, don’t do that.  If you did something else, please tell what it was :)
msg156318 - (view) Author: Alex Grönholm (alex.gronholm) * Date: 2012-03-19 05:39
>If I understand correctly that you used “/usr/bin/python pysetup install spam” and wanted it to install to /usr/lib/python2.7/site-packages, then I think that the correct reply is: Not supported, don’t do that.  If you did something else, please tell what it was :)

Did something along the lines of "pysetup install sqlalchemy" and it's trying to install it under /usr/lib/python2.7/site-packages, which is of course wrong. But since it ignores the customized site-local configuration, how could it possibly know where to install?
msg156467 - (view) Author: Alex Grönholm (alex.gronholm) * Date: 2012-03-20 22:07
>> Ignoring site-local configuration
>What do you mean?  sitecustomize is executed, for example.

Whatever, but it only looks for the paths in the included sysconfig.cfg. If the system Python uses a different sort of path in site.py, distutils2 won't use it.
History
Date User Action Args
2022-04-11 14:57:28adminsetgithub: 58564
2014-03-13 02:23:44eric.araujosetstatus: open -> closed
resolution: out of date
stage: resolved
2012-03-20 22:07:03alex.gronholmsetmessages: + msg156467
2012-03-19 05:39:12alex.gronholmsetmessages: + msg156318
2012-03-18 23:52:54eric.araujosetnosy: + barry
messages: + msg156294
2012-03-18 18:50:35alex.gronholmsetmessages: + msg156265
2012-03-18 11:44:55alexissetmessages: + msg156247
2012-03-18 06:10:34alex.gronholmcreate