With the release of Xcode 4.3 for OS X 10.7, Apple has moved the location of the OS X SDKs from their long-time path of /Developer to within the new Xcode.app bundle itself. This breaks the building of extension modules with any of Distutils or packaging/Distutils2 when Python was built as as universal build with an SDK in the former location, as is the case with the python.org installers.

A workaround is to either leave the old /Developer directory in place when upgrading to Xcode 4.3 or to install a symlink to the new location:

sudo ln -s /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer /Developer

maybe you can use xcode-select to set the correct path

xcode-select -print-path
/Applications/Xcode.app/Contents/Developer

Usage: xcode-select -print-path
or: xcode-select -switch <xcode_folder_path>

-switch <xcode_folder_path> Sets the path for the current Xcode

for detail, you can man xcode-select

s7v7nislands: you cannot use Xcode-select to fix this, xcode-select is used to switch between 2 or more installed versions of Xcode.

The issue is more annoying than the change of location of the SDK, the path to the compiler has also changed unless users manually install the Unix command-line tools, either using a button in the GUI or by installing a separate DMG.

There seem to be two options to thoroughly fixing this:

1. Tell users they must install the unix tools (cltools*.dmg)

   I'm not 100% sure at this time that this would work, from what I've
   learned from inspecting the installer packages this seems to imply
   "-isysroot /" (that is, no /Developer/SDKS/....)

   A small change to distutils and packaging would switch distutils from
   using "-isysroot /Developer/SDKs/..." to "-isysroot /" when it
   notices that the system root doesn't exist. This is similar to
   existing hacks for changing a number of compiler flags on OSX 10.3
   when using the 32-bit universal build.

2. Tweak the configure script, distutils and packaging to look for the
   SDK and compiler in the new location if they cannot be found in the
   default location.

   This would be the most user-friendly as the build would "just work",
   but is more work on our end. I have a 10.8 VM with and Xcode install
   where I haven't installed the unix tools to experiment with this.

   This is definitely more work than running configure with a
   different SDK and custom values for CC and CXX, we also have to
   convince configure that it isn't performing a cross-build (it seems
   that some compile checks are performed before we add -isysroot to
   CFLAGS and because I don't have the unix tools installed this results
   in a compile failure early in configure)

   This option might be of limited use though: when you want to build
   an extension that uses a C library that isn't shipped with the OS
   (for example PIL with libjpeg) the build of the python extension
   would "just work", but you'd still have to manually specify the
   compiler and system root for the build of libjpeg.

I've been working on this and it does need to be thoroughly fixed. There are two different aspects to it: (1) being able to build Python using any of the supported development environment options; and (2) support in Distutils and packaging to build extension modules for any of the supported development options. With the release of the Command Line Tools for 10.7 and beyond (and with the dangling mess of 10.6 with both Xcode 3.2.6 and Xcode 4.2 installations out there), we need to be able to support multiple options for users and builders of universal builds. Apple does supply tools to deal with development environments installed in arbitrary locations (primarily, xcode-select and xcrun) but there are some gaps in them for supporting command-line-based builds, like a convenient way to find the absolute path to the SDK. I'm working on patches for all of this right now. I'm hopeful we will end up with a major increase in usability to users of universal builds.

What I'd prefer to look for the compiler:

• in distutils: if $CC is an absolute path and exists, use that

• look for clang on $PATH, use it if found
  (default configure looks for GCC in preference of other compilers,
  but with Apple's toolchain it is better to use clang)

• look for gcc on $PATH, use it if found
  (needed to use the Fink compiler when Fink is on $PATH, alternative
  for 3.3 is to document that configure will try to pick and Apple
  compiler)

• use $(xcodebuild -find clang)

To look for the SDK:

• in distutils: if -isysroot is set, is not '/' and exists, use that

• in distutils: extract SDK version from -isysroot string (if set),
  then use $(xcode-select) to find the root of the installation and
  construct a path relative to that. If the requested SDK version exists,
  use that

• if osx release < 10.5 and building universal:

  • use /Developer/SDKs/MacOSX10.4u.sdk
    (needed for universal builds on early releases of OSX 10.4)

• if /usr/include/stdint.h exists: use -isysroot /
  (older Xcode, or Xcode >=4.3 with Command Line Tools installed)

• use $(xcodebuild -showsdks) to find list of SDKs and use most recent
  one.

I haven't tried to capture this in code yet, and haven't tested the procedure on earlier releases (or even a 10.7 system with Xcode 4.3 without unix tools), but this should work.

An initial set of changes to support Xcode 4.3+ have been applied for Python 3.3.0b1 have been applied via bpo-13590. As noted there, a back port of some subset is needed for 3.2 and 2.7. I'll update this after 3.3.0b1 has been released.

New changeset 73532f33fbb3 by Ned Deily in branch 'default':
Issue bpo-14499: Fix several problems with OS X universal build support:
http://hg.python.org/cpython/rev/73532f33fbb3

Not critical anymore. Probably safe to close?

Update: as of 3.6.5 and 2.7.15 and in the upcoming 3.7.0 release, macSO framework builds with --enable-universalsdk=/ no longer add -isysroot=/ to compiler and linker calls. They are not necessary and made it more difficult to attempt to override. Although we still do not officially support building on newer macOS systems for older systems (due to lack of "weak link" testing support and issues with setup.py and configure), it should now be possible to more easily use the Apple build tool chain features documented for xcrun, like the DEVELOPER_RUN and SDKROOT environment variables (see 'man xcrun'). This includes current python.org macOS installer builds. So I think we can finally close this.