This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: ensurepip and pip install failures on macOS Sierra with non-system Python 2.7.x
Type: Stage: patch review
Components: macOS Versions: Python 3.7, Python 3.6, Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: Marc.Culler, benjamin.peterson, ned.deily, python-dev, ronaldoussoren
Priority: Keywords: patch

Created on 2016-10-14 13:09 by Marc.Culler, last changed 2022-04-11 14:58 by admin.

File name Uploaded Description Edit
pipfails.patch Marc.Culler, 2016-10-14 13:08 patch
Pull Requests
URL Status Linked Edit
PR 5445 merged ned.deily, 2018-01-30 09:52
Messages (8)
msg278649 - (view) Author: Marc Culler (Marc.Culler) * Date: 2016-10-14 13:08
Changes made to /Library/Python on macOSX Sierra cause the --with-ensurepip compiler flag to fail, and lead to failures of pip after installing Python.

The new file that causes the problem on Sierra is:
The current version of reads that .pth file, which results in sys.path containing the path:
The latter directory (which is protected by SIP) contains many Python 2.7 packages, including easy_install, setuptools, six, py2app, numpy, pylab and pyOpenSSL.  The effect of including this SIP-protected path in sys.path is this:  installing or upgrade a package in /Library/Frameworks/Python.framework which is also installed as an "Extra" in the /System/Frameworks/Python.framework will cause pip to attempt to delete the "old" package from its SIP-protected directory, leading to a "Permission Denied" exception and a failed install.

Given that Apple has now tied /Library/Python to the system Python in this way, thereby making a separate Python framework in /Library/Frameworks unusable, the natural solution to this problem would be to stop including any /Library/Python paths in sys.path.

I am attaching a patch that removes the block of code in which adds these paths.
msg278657 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-10-14 16:54
This also affects the -m ensurepip installation of pip itself: the pip install fails trying to upgrade the Apple-supplied version of setuptools.  The behavior of adding the system Python's site-packages directory to the search path of all framework-build Pythons was deliberately added as a result of Issue4865.  I think experience has shown this was not a good idea because of the coupling it introduced between separate Python installations and some third-party distributors of Python on OS X already patch this code out.  It's currently only an issue for 2.7.x since Apple has not yet shipped versions of Python 3.x but the code should be removed there, too.
msg279779 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2016-10-31 11:26
Another reason to remove this feature: installing python 2.7.12 using the installer on breaks the system install of Python, likely because of this issue.

I'm a bit sad that this feature has to go, but modern Python use (in particular virtualenv) has reduced the need for this feature.
msg282272 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-12-03 07:15
New changeset a8a342b3fbc7 by Ned Deily in branch '2.7':
Issue #28440: No longer add /Library/Python/site-packages, the Apple-supplied
msg282275 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-12-03 07:21
I've pushed the change to revert Issue4865 for the 2.7 branch for release with 2.7.13.  I am leaving this open to push similar changes for 3.x branches in anticipation of Apple someday supplying system Python 3.x.  It would also be good to see if the post-install script that runs ensurepip can be made to fail more noisily.
msg311261 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-01-30 10:14
New changeset 763f094b1f0ce2a07768828e78afe9687e9ad3bb by Ned Deily in branch 'master':
bpo-28440: Don't add /Library/Python/3.x/site-packages to sys.path (#5445)
msg347833 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2019-07-13 14:45
In response to msg282272: That day has come, the beta for 10.15 contains Python 3.7.3.
msg347840 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-07-13 15:23
> That day has come, the beta for 10.15 contains Python 3.7.3.

Yay!  FTR, 763f094b1f0ce2a07768828e78afe9687e9ad3bb was released in 3.7.0.

The only reason I have left this issues open was that I also wanted to make sure that ensurepip failures in the post-install script do not fail silently.
Date User Action Args
2022-04-11 14:58:38adminsetgithub: 72626
2019-07-13 15:23:04ned.deilysetmessages: + msg347840
2019-07-13 14:45:33ronaldoussorensetmessages: + msg347833
2018-01-30 10:14:14ned.deilysetmessages: + msg311261
2018-01-30 09:52:41ned.deilysetpull_requests: + pull_request5277
2016-12-03 07:21:14ned.deilysetpriority: release blocker ->

messages: + msg282275
versions: + Python 3.6, Python 3.7
2016-12-03 07:15:15python-devsetnosy: + python-dev
messages: + msg282272
2016-10-31 11:26:02ronaldoussorensetmessages: + msg279779
2016-10-14 16:54:39ned.deilysetpriority: normal -> release blocker

type: compile error ->
assignee: ned.deily

title: pip failures on macOS Sierra -> ensurepip and pip install failures on macOS Sierra with non-system Python 2.7.x
nosy: + benjamin.peterson
versions: + Python 2.7
messages: + msg278657
stage: patch review
2016-10-14 15:07:42r.david.murraysetnosy: + ronaldoussoren, ned.deily
components: + macOS
2016-10-14 13:09:00Marc.Cullercreate