This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Manually Installed Python Includes System Wide Paths
Type: behavior Stage:
Components: macOS Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: ronaldoussoren Nosy List: James.Kyle, eric.araujo, ned.deily, ronaldoussoren
Priority: normal Keywords:

Created on 2012-06-11 23:23 by James.Kyle, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (9)
msg162648 - (view) Author: James Kyle (James.Kyle) Date: 2012-06-11 23:23
This behavior is present on OS X 10.7 and framework builds.

In this case, the /Library/Python/<version> paths are included in every install. 

I would consider this behavior non-standard as in most manual python installs only that installations library paths are included.

This can lead to surprising and inconsistent behavior if multiple installs are present (very common on osx, e.g. macports + system install).

This originated as a macports bug ticket: https://trac.macports.org/ticket/34763
msg162649 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2012-06-11 23:25
This is intentional behavior, you can install packages you want to share between python installations in /Library/Python instead of the regular site-packages directory.

Macports could always patch their site.py file to avoid this.
msg162650 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2012-06-11 23:38
This was added in issue4865.

(The same behavior is present in 3.2 and 3.3)
msg162651 - (view) Author: James Kyle (James.Kyle) Date: 2012-06-11 23:50
Am I missing something or were the problems delineated in issue #4865 solvable by simply sys.path.append("/Library/Python/2.7/site-packages")?

What would the process be for reopening this issue for discussion? 

I'm not sure this is the right way to address this. For example, what if other *nix distros started adding their own custom "common" paths, would Python begin implemented these distro specific deviations from standard behavior?

In my honest opinion, there's an expectation of package isolation in independent python installs.
msg162652 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2012-06-11 23:59
Python installation are already not isolated: there is a per-user site-packages directory on all platforms that is shared between all installations of a particular python release. This directory is located in a subdirectory of ~/.local on POSIX systems (including OSX).

Anyway, this cannot be changed for a released version of python because that would break backward compatibility. At "best" this could be disabled in python 3.3 and even there I'm far from convinced that disabling this feature would be worthwhile.
msg162653 - (view) Author: James Kyle (James.Kyle) Date: 2012-06-12 00:06
Fair enough. Thanks!
msg162657 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2012-06-12 01:50
I have to admit that I'm not keen on this feature for the reasons James cited.  And I think the example of the shared user site directory is not a good analogy.  In that case, you, as a user, have more control over the presence and contents of the directory since it is located within your home directory.  You would normally need administrator privilege to manipulate /Library/Python.  And, in any case, it would be better if there *were* separate user site directories per Python instance, IMO.  Yes, you can play with sys.path after the fact but that's not very friendly.  It certainly can lead to confusion.  An uncontrived example:

$ sudo easy_install-2.7 appscript
[...]
Installed /Library/Python/2.7/site-packages/appscript-1.0.0-py2.7-macosx-10.7-intel.egg
$ /usr/bin/python2.7 -c "import appscript"
$ /usr/local/bin/python2.7 -c "import appscript"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "build/bdist.macosx-10.7-intel/egg/appscript/__init__.py", line 8, in <module>
  File "build/bdist.macosx-10.7-intel/egg/aem/__init__.py", line 5, in <module>
  File "build/bdist.macosx-10.7-intel/egg/aem/ae.py", line 7, in <module>
  File "build/bdist.macosx-10.7-intel/egg/aem/ae.py", line 3, in __bootstrap__
ImportError: No module named pkg_resources

... because the non-system Python did not have Distribute or setuptools installed, whereas Apple supplies it with the system Python.  It's not the only package to fail in a similar way. I expect there are other cases where differences in extension module builds could cause problems.  To me, the feature seems to go against "explicit is better than implicit".

As far as I know, the only place where this behavior is documented is in the What's New documents for 2.7.  I wouldn't object to removing it in 3.3.
msg162659 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2012-06-12 02:41
A few more thoughts.  The original impetus for this feature was Issue4865.  The use case there seem to be from users of Google App Engine back when it was released using Python 2.5. It seems to me that the use of dmg installers for Python packages has diminished; certainly the unofficial packages at pythonmac.org haven't been updated for more recent releases. The issues with installing some packages (like PIL) with 3rd-party C library dependencies notwithstanding, I wonder if part of the original call for this feature was the fact that Apple does provide setuptools easy_install's with system Pythons and there is confusion that a setuptools/Distribute instance, with its own easy_install command, is needed for each Python.  Some of that confusion should diminish over time with the availability of "pip -E" and, starting with 3.3, a batteries-included installer command, pysetup.

Also, with regard to backward compatibility, I speculate that there hasn't been much notice of this feature since it only affects users of Python 2.7 on OS X 10.7+.  For Python 3, the presence or absence of the feature doesn't affect anyone because Apple has yet to ship a system Python 3 so removing it from 3.3 would have no backward compatibility impacts, unless an administrator manually created a /Library/Python/3.x for some reason and manually installed things there.
msg162686 - (view) Author: James Kyle (James.Kyle) Date: 2012-06-12 16:17
I think Ned does have some good points regarding the minimal impact a reversion would have.

The most poignant point is that /Library/ on OS X is not a user controlled directory whereas ~/.local is. If ~/.local exists and has packages installed, it's because the user has made a conscious choice. If they regret that choice, they can remove ~/.local. 

Every package installed in ~/.local is the result of direct action by the user impacted. The same cannot be said for /Library.
History
Date User Action Args
2022-04-11 14:57:31adminsetgithub: 59253
2012-06-16 16:27:01eric.araujosetnosy: + eric.araujo
2012-06-12 16:17:26James.Kylesetmessages: + msg162686
2012-06-12 02:41:40ned.deilysetmessages: + msg162659
2012-06-12 01:50:37ned.deilysetmessages: + msg162657
2012-06-12 00:06:54James.Kylesetmessages: + msg162653
2012-06-11 23:59:57ronaldoussorensetnosy: + ned.deily
messages: + msg162652
2012-06-11 23:50:06James.Kylesetmessages: + msg162651
2012-06-11 23:38:34ronaldoussorensetstatus: open -> closed
2012-06-11 23:38:25ronaldoussorensetresolution: rejected
messages: + msg162650
versions: + Python 3.2, Python 3.3
2012-06-11 23:25:42ronaldoussorensetmessages: + msg162649
2012-06-11 23:23:02James.Kylecreate