Author larry
Recipients larry
Date 2009-04-23.08:25:34
SpamBayes Score 5.55112e-16
Marked as misclassified No
Message-id <1240475142.53.0.423262271042.issue5819@psf.upfronthosting.co.za>
In-reply-to
Content
The attached patch adds support for a new environment variable,
PYTHONPREFIXES.  PYTHONPREFIXES is similar to PYTHONUSERBASE: it lets
you add "prefix directories" to be culled for site packages.  It differs
from PYTHONUSERBASE in three ways:

* PYTHONPREFIXES has an empty default value.  PYTHONUSERBASE has a
  default, e.g. ~/.local on UNIX-like systems.

* PYTHONPREFIXES supports multiple directories, separated by the
  site-specific directory separator character (os.pathsep).   Earlier
  directories take precedence.  PYTHONUSERBASE supports specifying
  at most one directory.

* PYTHONPREFIXES adds its directories to site.PREFIXES, so it reuses
  the existing mechanisms for site package directories, exactly
  simulating a real prefix directory.  PYTHONUSERBASE only adds a
  single directory, using its own custom code path.


This last point bears further discussion.  PYTHONUSERBASE's custom code
to inspect only a single directory has resulted in at least one bug, if
not more, as follows:

* The bona-fide known bug: the Debian package mantainer for Python
  decided to change "site-packages" to "dist-packages" in 2.6,
  for reasons I still don't quite understand.  He made this change in
  site.addsitepackages and distutils.sysconfig.get_python_lib, and
  similarly in setuptools, but he missed changing it in
  site.addusersitepackages.  This meant that if you used setup.py to
  install a package to a private prefix directory, PYTHONUSERBASE had
  no hope of ever finding the package.  (Happily this bug is fixed.)

* I suspect there's a similar bug with PYTHONUSERBASE on the "os2emx"
  and "riscos" platforms.  site.addsitepackages on those platforms
  looks in "{prefix}/Lib/site-packages", but
  site.addusersitepackages looks in
  "{prefix}/lib/python{version}/site-packages" as it does
  on any non-Windows platform.  Presumably setup.py on those two
  platforms installs site packages to the directory site.addsitepackages
  inspects, which means that PYTHONUSERBASE doesn't work on those
  two platforms.

PYTHONUSERBASE's custom code path to add site package directories is a
source of unnecessary complexity and bugs.  I cannot fathom why its
implementors chose this approach; in any case I think reusing
site.addsitepackages is a clear win.  I suspect it's too late to change
the semantics of PYTHONUSERBASE to simply call site.addsitepackages,
though if that idea found support I'd be happy to contribute a patch.


A few more notes on PYTHONPREFIXES:

* PYTHONPREFIXES is gated by the exact same mechanisms that shut off
  PYTHONUSERBASE.
    * Specifying "-s" on the Python command line shuts it off.
    * Setting the environment variable PYTHONNOUSERSITE to a non-empty
      string shuts it off.
    * If the effective uid / gid doesn't match the actual uid / gid it
      automatically shuts off.

* I'm not enormously happy with the name.  Until about an hour or two
  ago I was calling it "PYTHONUSERBASES".  I'm open to other
  suggestions.

* I'm not sure that PYTHONPREFIX should literally modify site.PREFIXES.
  If that's a bad idea I'd be happy to amend the patch so it didn't
  touch site.PREFIXES.

* Reaction in python-ideas has been reasonably positive, though I gather
  Nick Coughlan and Scott David Daniels think it's unnecessary.  (To
  read the discussion, search for the old name: "PYTHONUSERBASES".)

* Ben Finney prefers a counter-proposal he made in the python-ideas
  discussion: change the existing PYTHONUSERBASE to support multiple
  directories.  I don't like this approach, because:
    a) it means you have to explicitly add the local default if you
       want to use it, and
    b) PYTHONUSERBASE only inspects one directory, whereas PYTHONPREFIX
       inspects all the directories Python might use for site packages.
  I do admit this approach would be preferable to no change at all.


The attached patch is thrillingly simple and works fine.  However it's
not ready to be merged because I haven't touched the documentation.  I
figured I'd hold off until I see which way the wind blows.

I'd also be happy to whip out a PEP if that's what is called for.
History
Date User Action Args
2009-04-23 08:25:43larrysetrecipients: + larry
2009-04-23 08:25:42larrysetmessageid: <1240475142.53.0.423262271042.issue5819@psf.upfronthosting.co.za>
2009-04-23 08:25:40larrylinkissue5819 messages
2009-04-23 08:25:37larrycreate