The implementation of pep-0370 treats OSX like any other unix platform. 

This is problemantic for two reasons: first of all OSX already had a per-user directory before pep-0370 was implement: ~/Library/Python/X.Y, which means there are now two per-user directories on OSX. Secondly the pep-0370  per-user directory does not honor platform conventions.

I therefore propose to change pep-0370 behavior on OSX: 
* remove ~/.local as the user-base
* upgrade ~/Library/Python/X.Y to a pep-0370 compatible directory
  (that is: introduce the additional subdirectories that are used
   in pep-0370)
* upgrade /Library/Python/X.Y to the same structure

The attached patch is untested an implements the proposed behavior for framework builds, unix builds would keep the unix-style behavior (and would lose access to ~/Library/Python).

The patch is slightly complicated by adding support for --with-framework-name as well, but that shouldn't make the patch harder to read.

BTW. The "needs review" part is more for the proposed change in behavior that for the actual patch, I'm confident that I can create a correct patch but I'm not sure if the change is behavior is uncontroversial.

Since this needs review, and Christian is the author of that PEP, I'm assigning it to him.  If not appropriate, any suggestions on where to get visibility to get reviewers?

@Sean: you can look at the maintainers.rst file in the py3k branch I guess

I am maintaining sysconfig, so I guess I'll just help on the review.

Notice that we might need to backport some of the work in distutils/sysconfig since we have decided to revert distutils to 2.6/3.1 state in 2.7/3.2. I'll wait for Christian reviews, and make sure that this part is OK

I'd prefer to have this resolved before the next 2.7 beta.

Since you want it before the next beta, I'm promoting it to release blocker.  Benjamin can reduce it if needed when we get down to the wire.

Christian hasn't been around for a while, and I'd like to release the next beta on time. Thus, I think Tarek's or my review will be sufficent. The idea seems fine to me.

Committed to the trunk in r80967.

Reducing the priority because this is no longer a release-blocker for the next 2.7 release.

And committed to 3.2 in r80968

> And committed to 3.2 in r80968

This commit introduced a regression:

http://www.python.org/dev/buildbot/all/builders/AMD64 Ubuntu wide 3.x/builds/1055

======================================================================
FAIL: test_get_scheme_names (test.test_sysconfig.TestSysConfig)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/cpython-ucs4-nonascii-€/3.x.pitrou-ubuntu-wide/build/Lib/test/test_sysconfig.py", line 239, in test_get_scheme_names
    self.assertEquals(get_scheme_names(), wanted)
AssertionError: Tuples differ: ('nt', 'nt_user', 'os2', 'os2_... != ('nt', 'nt_user', 'os2', 'os2_...

First differing element 4:
osx_framework_user
posix_home

First tuple contains 1 additional elements.
First extra element 7:
posix_user

  ('nt',
   'nt_user',
   'os2',
   'os2_home',
-  'osx_framework_user',
   'posix_home',
   'posix_prefix',
   'posix_user')

Fixed test_sysconfig.py in r80986.

Ronald, r80967 makes test_site fail on my machine, with the following output:

======================================================================
FAIL: test_getsitepackages (__main__.HelperFunctionsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_site.py", line 188, in test_getsitepackages
    self.assertEqual(len(dirs), 4)
AssertionError: 2 != 4

----------------------------------------------------------------------
Ran 17 tests in 0.321s

FAILED (failures=1)
Traceback (most recent call last):
  File "Lib/test/test_site.py", line 333, in <module>
    test_main()
  File "Lib/test/test_site.py", line 330, in test_main
    run_unittest(HelperFunctionsTests, ImportSideEffectTests)
  File "/Users/dickinsm/python/svn/trunk/Lib/test/test_support.py", line 1038, in run_unittest
    _run_suite(suite)
  File "/Users/dickinsm/python/svn/trunk/Lib/test/test_support.py", line 1021, in _run_suite
    raise TestFailed(err)
test.test_support.TestFailed: Traceback (most recent call last):
  File "Lib/test/test_site.py", line 188, in test_getsitepackages
    self.assertEqual(len(dirs), 4)
AssertionError: 2 != 4

Any ideas why?  This is on a non-framework debug build of the trunk.  The variable 'dirs' turns out to be:

['Python.framework/lib/python2.7/site-packages', 'Python.framework/lib/site-python']

What else should be there?

I'll look into this in the morning.

It fails on "x86 Tiger trunk" buildbot, too.
(the PPC Tiger buildbot is happy)

The tests no longer fail.

Would it be possible to have this reverted for 2.7.2 / 3.2.1, and restore ~/.local _and_ ~/Library as equally valid locations for python code?

As I tried to explain on the mailing list, this change basically introduces a regression, and nothing else.  Sorry I did not comment on the ticket at the time.

Previously it was possible to maintain a single "~/.pydistutils.cfg" for UNIX-like OSes, which would allow 'python setup.py install', 'pip install' and 'easy_install' to function without adding a 'sudo' in front of them, or any command-line options.  This is important since easy_install doesn't expose a '--user'.  Now, there needs to be a specific, separate config file for OS X installs.

Also, it was previously possible to pass a sane value for '--prefix', but now there is no such value.  This makes it much more difficult to, for example:

  cd my-c-dependency
  ./configure --prefix ~/.local
  cd my-python-project
  python setup.py install --prefix ~/.local

since Python code with C dependencies will sometimes attempt to discover the location of include files and the like via the install prefix.

'setup.py install --user' (the only remaining installation mechanism which actually works out of the box on a mac) will now place scripts into ~/Library/Python/x.y/bin, which seems a singularly unhelpful location.  Again, if there are C and Python programs which interact, I now need 2 locations on my $PATH instead of one.

I don't understand what this change buys anyone.  If you're treating the Mac python as a UNIX-like environment, it just spuriously breaks things.  But if you're treating it as a Mac-like environment (i.e. not using a terminal), the right place to put your Python code is _inside a (framework or application) bundle_, not lying around your home directory.

Also, if there is interest in properly honoring Mac filesystem conventions, there are APIs for doing that, as documented here: <http://developer.apple.com/library/mac/#documentation/MacOSX/Conceptual/BPFileSystem/Articles/Domains.html>.  Manually enumerating the user domain while ignoring, for example, the network domain, seems half-hearted and arbitrary.

(reopening to ensure glyph's message doesn't get lost)

It would be nice to have feedback from the Mac experts on this.

Please mention Apple's OS X Python behavior in the PEP. The Python that comes with OS X Lion doesn't seem to follow the PEP regarding ~/.local ; this deserves a mention.

It seems that the boat has sailed here

For what it's worth, I've long since adapted to the status quo here, so I'm happy for this to just be closed :-).

(Apple seems to have mostly given up on "domains" as a user-facing or even developer-facing concept, so searching /Network seems like it is not relevant any more.  /System is now read-only even for root and they're deprecating the Python installed there; still, the up-to-date reference for the relevant API is https://developer.apple.com/documentation/foundation/nsfilemanager/1407726-urlsfordirectory?language=objc# )