classification
Title: system wide site-packages dir not used on Mac OS X
Type: behavior Stage:
Components: Library (Lib), Macintosh Versions: Python 3.1, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ronaldoussoren Nosy List: kapet, loewis, ronaldoussoren
Priority: normal Keywords: patch

Created on 2009-01-07 12:51 by kapet, last changed 2009-03-30 23:16 by ronaldoussoren. This issue is now closed.

Files
File name Uploaded Description Edit
apple.diff kapet, 2009-01-07 12:50 Diff stock Python 2.5.1 against Apple Python
Messages (5)
msg79329 - (view) Author: Karsten Petersen (kapet) Date: 2009-01-07 12:50
[Guido asked me to file this as an issue.]

On Mac OS X Apple has configured the system provided Python to also
search this global site-packages directory:
   /Library/Python/<version>/site-packages/

This allows users to install their own system-wide packages there (and
not have them to drop into /System/Library/Frameworks/Python....../).

Unfortunately if you install Python from python.org (or pythonmac.org)
it is not searching this path, it only searches the site-packages
directory _within_ its own framework:
  /Library/Frameworks/Python...../
That directory is of course not searched by the Apple provided Python.

This is an issue for addon package installers (think PIL or numpy): 
There is no single path that will be found by both the system provided
Python and a Python installed from python.org.  This makes most/all (?)
of the pre-built packages on pythonmac.org unusable with the version of
Python Apple ships although there is no version conflict or such - it
simply installs stuff in the "wrong" directories.

Many people who tried to install PIL for Django or App Engine stumbled
into this.  The system provided Python version was fine.  But compiling
PIL is hard.  So they installed PIL from pythonmac.org.  That couldn't
be found by Python.  Now some people manually copied stuff around, some
created symlinks, some installed a second Python 2.5.  (The lucky ones
picked a Python download from python.org, the not so lucky ones
installed the 2.5.0 release from pythonmac.org...)

I've attached a diff of a stock Python 2.5.1 library against what I
found on my Apple Mac OS X 10.5 (Leopard) system.  I've dropped changes
that are not related to the path.
msg79352 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-01-07 17:41
IIUC, you are not the author of the patch (in the sense that you wrote
the new code), but rather created a diff between Apple's code and
Python's. In this case, we cannot accept the patch - it needs to be
contributed by the authors (i.e. Apple).

Furthermore, I fail to see how this patch can affect the places were
Python searches for site-packages - it only deals with Python code that
might need adjustment if Python would search in this other location.

Please correct me if I'm wrong.
msg79355 - (view) Author: Karsten Petersen (kapet) Date: 2009-01-07 17:58
Sorry, I should have made this clear:  I am NOT proposing to add this
patch.  I meant this more to show what related changes were done by
Apple.  Some of these clearly wouldn't even work like this if applied to
a Python not installed in /System/Library/Frameworks/...

IMHO the most interesting piece of that diff is the change to site.py
which makes it look into /Library/Python/
msg84552 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-03-30 14:37
I'm in favour of adding /Library/Python/x.y/ to sys.path for Python 2.7 
and 3.1 and will work on that during the Pycon sprints.

/Library/Python will be added after the site-packages directory inside 
the framework instead of replacing the latter directory. The reason for 
this is that we don't want to install all packages in a location that's 
visible to Apple's copy of Python because you could accidently replace 
packages that Apple ships as part of the "Extras" directory that way. 
Replacing those is bad because those libraries are used by Apple system 
software and replacing stuff might therefore break the system 
(particularly on server systems).

I won't backport to 2.6 and 3.0 because this is a feature request and 
not a bugfix.

W.r.t. building PIL and other extensions: we've had a discussion about 
this during the startup phase of the macpython-sig sprint and we have 
some ideas to make this situation less painful in the future (without 
necessarily having to patch Python for that).
msg84711 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-03-30 23:16
Committed a fix for this as r70778 (trunk), r70782 (3.1)
History
Date User Action Args
2009-03-30 23:16:33ronaldoussorensetstatus: open -> closed
resolution: fixed
messages: + msg84711
2009-03-30 14:37:58ronaldoussorensetversions: + Python 3.1, Python 2.7
nosy: + ronaldoussoren

messages: + msg84552

assignee: ronaldoussoren
2009-01-07 17:58:35kapetsetmessages: + msg79355
2009-01-07 17:41:09loewissetnosy: + loewis
messages: + msg79352
2009-01-07 12:51:01kapetcreate