classification
Title: pep-0370 on osx duplicates existing functionality
Type: behavior Stage:
Components: Installation, Library (Lib), Macintosh Versions: Python 3.3, Python 3.2, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, christian.heimes, dholth, eric.araujo, flox, glyph, haypo, jafo, mark.dickinson, ned.deily, r.david.murray, ronaldoussoren, tarek
Priority: normal Keywords:

Created on 2010-03-07 12:29 by ronaldoussoren, last changed 2012-06-07 13:27 by dholth.

Files
File name Uploaded Description Edit
issue8084.patch ronaldoussoren, 2010-03-07 12:33
Messages (19)
msg100577 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-03-07 12:29
The implementation of pep-0370 treats OSX like any other unix platform. 

This is problemantic for two reasons: first of all OSX already had a per-user directory before pep-0370 was implement: ~/Library/Python/X.Y, which means there are now two per-user directories on OSX. Secondly the pep-0370  per-user directory does not honor platform conventions.

I therefore propose to change pep-0370 behavior on OSX: 
* remove ~/.local as the user-base
* upgrade ~/Library/Python/X.Y to a pep-0370 compatible directory
  (that is: introduce the additional subdirectories that are used
   in pep-0370)
* upgrade /Library/Python/X.Y to the same structure
msg100578 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-03-07 12:33
The attached patch is untested an implements the proposed behavior for framework builds, unix builds would keep the unix-style behavior (and would lose access to ~/Library/Python).

The patch is slightly complicated by adding support for --with-framework-name as well, but that shouldn't make the patch harder to read.

BTW. The "needs review" part is more for the proposed change in behavior that for the actual patch, I'm confident that I can create a correct patch but I'm not sure if the change is behavior is uncontroversial.
msg101387 - (view) Author: Sean Reifschneider (jafo) * (Python committer) Date: 2010-03-20 18:36
Since this needs review, and Christian is the author of that PEP, I'm assigning it to him.  If not appropriate, any suggestions on where to get visibility to get reviewers?
msg101410 - (view) Author: Tarek Ziadé (tarek) * (Python committer) Date: 2010-03-20 23:29
@Sean: you can look at the maintainers.rst file in the py3k branch I guess

I am maintaining sysconfig, so I guess I'll just help on the review.

Notice that we might need to backport some of the work in distutils/sysconfig since we have decided to revert distutils to 2.6/3.1 state in 2.7/3.2. I'll wait for Christian reviews, and make sure that this part is OK
msg104491 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-04-29 12:17
I'd prefer to have this resolved before the next 2.7 beta.
msg104538 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-04-29 16:54
Since you want it before the next beta, I'm promoting it to release blocker.  Benjamin can reduce it if needed when we get down to the wire.
msg105219 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2010-05-07 18:03
Christian hasn't been around for a while, and I'd like to release the next beta on time. Thus, I think Tarek's or my review will be sufficent. The idea seems fine to me.
msg105274 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-05-08 10:30
Committed to the trunk in r80967.

Reducing the priority because this is no longer a release-blocker for the next 2.7 release.
msg105275 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-05-08 10:50
And committed to 3.2 in r80968
msg105281 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-05-08 11:20
> And committed to 3.2 in r80968

This commit introduced a regression:

http://www.python.org/dev/buildbot/all/builders/AMD64 Ubuntu wide 3.x/builds/1055

======================================================================
FAIL: test_get_scheme_names (test.test_sysconfig.TestSysConfig)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/cpython-ucs4-nonascii-€/3.x.pitrou-ubuntu-wide/build/Lib/test/test_sysconfig.py", line 239, in test_get_scheme_names
    self.assertEquals(get_scheme_names(), wanted)
AssertionError: Tuples differ: ('nt', 'nt_user', 'os2', 'os2_... != ('nt', 'nt_user', 'os2', 'os2_...

First differing element 4:
osx_framework_user
posix_home

First tuple contains 1 additional elements.
First extra element 7:
posix_user

  ('nt',
   'nt_user',
   'os2',
   'os2_home',
-  'osx_framework_user',
   'posix_home',
   'posix_prefix',
   'posix_user')
msg105308 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2010-05-08 16:12
Fixed test_sysconfig.py in r80986.
msg105330 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-05-08 19:46
Ronald, r80967 makes test_site fail on my machine, with the following output:

======================================================================
FAIL: test_getsitepackages (__main__.HelperFunctionsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_site.py", line 188, in test_getsitepackages
    self.assertEqual(len(dirs), 4)
AssertionError: 2 != 4

----------------------------------------------------------------------
Ran 17 tests in 0.321s

FAILED (failures=1)
Traceback (most recent call last):
  File "Lib/test/test_site.py", line 333, in <module>
    test_main()
  File "Lib/test/test_site.py", line 330, in test_main
    run_unittest(HelperFunctionsTests, ImportSideEffectTests)
  File "/Users/dickinsm/python/svn/trunk/Lib/test/test_support.py", line 1038, in run_unittest
    _run_suite(suite)
  File "/Users/dickinsm/python/svn/trunk/Lib/test/test_support.py", line 1021, in _run_suite
    raise TestFailed(err)
test.test_support.TestFailed: Traceback (most recent call last):
  File "Lib/test/test_site.py", line 188, in test_getsitepackages
    self.assertEqual(len(dirs), 4)
AssertionError: 2 != 4

Any ideas why?  This is on a non-framework debug build of the trunk.  The variable 'dirs' turns out to be:

['Python.framework/lib/python2.7/site-packages', 'Python.framework/lib/site-python']

What else should be there?
msg105337 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-05-08 20:24
I'll look into this in the morning.
msg105644 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-05-13 16:27
It fails on "x86 Tiger trunk" buildbot, too.
(the PPC Tiger buildbot is happy)
msg108795 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-06-27 14:44
The tests no longer fail.
msg122935 - (view) Author: Glyph Lefkowitz (glyph) Date: 2010-11-30 21:09
Would it be possible to have this reverted for 2.7.2 / 3.2.1, and restore ~/.local _and_ ~/Library as equally valid locations for python code?

As I tried to explain on the mailing list, this change basically introduces a regression, and nothing else.  Sorry I did not comment on the ticket at the time.

Previously it was possible to maintain a single "~/.pydistutils.cfg" for UNIX-like OSes, which would allow 'python setup.py install', 'pip install' and 'easy_install' to function without adding a 'sudo' in front of them, or any command-line options.  This is important since easy_install doesn't expose a '--user'.  Now, there needs to be a specific, separate config file for OS X installs.

Also, it was previously possible to pass a sane value for '--prefix', but now there is no such value.  This makes it much more difficult to, for example:

  cd my-c-dependency
  ./configure --prefix ~/.local
  cd my-python-project
  python setup.py install --prefix ~/.local

since Python code with C dependencies will sometimes attempt to discover the location of include files and the like via the install prefix.

'setup.py install --user' (the only remaining installation mechanism which actually works out of the box on a mac) will now place scripts into ~/Library/Python/x.y/bin, which seems a singularly unhelpful location.  Again, if there are C and Python programs which interact, I now need 2 locations on my $PATH instead of one.

I don't understand what this change buys anyone.  If you're treating the Mac python as a UNIX-like environment, it just spuriously breaks things.  But if you're treating it as a Mac-like environment (i.e. not using a terminal), the right place to put your Python code is _inside a (framework or application) bundle_, not lying around your home directory.

Also, if there is interest in properly honoring Mac filesystem conventions, there are APIs for doing that, as documented here: <http://developer.apple.com/library/mac/#documentation/MacOSX/Conceptual/BPFileSystem/Articles/Domains.html>.  Manually enumerating the user domain while ignoring, for example, the network domain, seems half-hearted and arbitrary.
msg123563 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-12-07 16:37
(reopening to ensure glyph's message doesn't get lost)
msg140662 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-07-19 13:10
It would be nice to have feedback from the Mac experts on this.
msg162472 - (view) Author: Daniel Holth (dholth) (Python committer) Date: 2012-06-07 13:27
Please mention Apple's OS X Python behavior in the PEP. The Python that comes with OS X Lion doesn't seem to follow the PEP regarding ~/.local ; this deserves a mention.
History
Date User Action Args
2012-06-07 13:27:00dholthsetnosy: + dholth
messages: + msg162472
2011-07-19 13:11:10eric.araujounlinkissue10881 superseder
2011-07-19 13:10:21eric.araujosetassignee: christian.heimes ->
versions: + Python 3.3
keywords: - patch, needs review, buildbot
nosy: + ned.deily

messages: + msg140662
resolution: fixed ->
stage: committed/rejected ->
2011-01-10 22:08:37eric.araujosetnosy: + eric.araujo
2011-01-10 21:49:17ixokailinkissue10881 superseder
2010-12-07 16:37:10ronaldoussorensetstatus: closed -> open

messages: + msg123563
2010-11-30 21:09:15glyphsetnosy: + glyph
messages: + msg122935
2010-06-27 14:44:08ronaldoussorensetstatus: open -> closed

messages: + msg108795
2010-05-13 16:27:40floxsetkeywords: + buildbot
nosy: + flox
messages: + msg105644

2010-05-08 20:24:37ronaldoussorensetmessages: + msg105337
2010-05-08 19:46:35mark.dickinsonsetstatus: closed -> open
nosy: + mark.dickinson
messages: + msg105330

2010-05-08 16:12:59benjamin.petersonsetstatus: open -> closed
resolution: fixed
messages: + msg105308
2010-05-08 11:20:04hayposetstatus: closed -> open

nosy: + haypo
messages: + msg105281

resolution: fixed -> (no value)
2010-05-08 10:50:15ronaldoussorensetstatus: open -> closed
resolution: fixed
messages: + msg105275

stage: committed/rejected
2010-05-08 10:30:01ronaldoussorensetpriority: release blocker -> normal

messages: + msg105274
2010-05-07 18:03:25benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg105219
2010-04-29 16:54:15r.david.murraysetpriority: high -> release blocker
nosy: + r.david.murray
messages: + msg104538

2010-04-29 12:33:12tareksetpriority: normal -> high
2010-04-29 12:17:58ronaldoussorensetmessages: + msg104491
2010-03-20 23:29:58tareksetmessages: + msg101410
2010-03-20 23:21:29pitrousetnosy: + tarek
2010-03-20 18:36:19jafosetpriority: normal

nosy: + christian.heimes, jafo
messages: + msg101387

assignee: ronaldoussoren -> christian.heimes
2010-03-07 12:33:47ronaldoussorensetfiles: + issue8084.patch
keywords: + patch
messages: + msg100578
2010-03-07 12:29:09ronaldoussorencreate