This bug is a crosspost from /setuptools.
(http://bugs.python.org/setuptools/issue122)

Summary:
OSX 10.6.x with Xcode 4 installed. Xcode 4 removes the PPC assembler from GCC. I attempted to install Fabric-1.0.0 which depended on pycrypto, during the pycrypto build there was a failure regarding "Broken Pipe".

I downloaded the package separately and attempted to do the install there. Same error.

I was pointed in the right direction that the broken pipe error was coming from the '-arch ppc' flag during build.


Expected Result
10.6.x should not report itself as "universal" when it's no longer possible to compile for PPC.


Actual Result
Attempted PPC compilation fails.


More detail:
http://superuser.com/questions/259278/python-2-6-1-pycrypto-2-3-pypi-package-broken-pipe-during-build/260106#260106

Let me know if you require more information.

Kind Regards,
Johannes

Thanks for the report.  Can you try to reproduce the bug with a distutils setup script to confirm the bug and get an error log?  You may have to apply the patch from #11599 to get a complete error message.  (Also, superuser rights are not required for build, so let’s remove that too.)

I only know the basics about C compilation, and I only know that Mac OS is a strange and peculiar system, but let’s try to work on this bug :)  The distutils.unixccompiler module already has code to work around things in various Mac OS versions, so if there is an easy way to detect “when it’s no longer possible to compile for PPC”, there is hope.

The release of Xcode 4 for Mac OS X 10.6 has complicated things for Python builds.  We need to sort out what the problems are and possible solutions for them.  I suggest this issue be reassigned to Ronald and me.

In the meantime, the workaround is to continue to use Xcode 3 (which is still fully supported) on OS X 10.6.  If you feel you absolutely need to have Xcode 4 installed *and* you still need to build C extension modules for the currently available Pythons for OS X (i.e. the python.org installers or even the Apple-supplied Pythons in 10.60), there are unofficial suggestions out on the web on how one can have Xcode 3 and 4 installed side-by-side, for example:
   http://hints.macworld.com/article.php?story=20110318050811544
(Note, I have not tried this and am not recommending it.)

Issue11485 is related to this.

As Ned noted Ned and I will look into this.

Thanks for taking care of this.  I’ll be here to review distutils-related changes, if any are needed.

I haven't put much thought in a solution yet, but at this point I'd go for three changes:

1) Give a clear warning when python was configured for i386/ppc 
   and Xcode4 is installed (instead of giving a vague compiler crash 
   due to crapping out on ppc code)

2) Add an option that forces distutils to compile for i386 only

   This should not be the default because that would result in broken
   packages for users that want to redistribute binaries to ppc 
   machines.

3) The i386/ppc build should use the system headers instead of the
   10.4u SDK (change in configure.in), with a fallback to the 10.4u
   SDK when building extensions on a 10.4 machine (because the 
   "-isysroot /" cannot build universal binaries on all such machines)

BTW. distutils does not have code to work around OSX issues, it has code
to make use of OSX specific compilation features (such as having universal binaries and SDKs). ;-) ;-)

A similar issue on StackOverflow reminded me that Distutils has support for the ARCHFLAGS environment variable on Mac OS X which you can use when building a C extension module to override the ARCH values that Python was built with.  I don't have Xcode 4 installed yet to verify this but you may be able to work around the problem by something like:

ARCHFLAGS='-arch i386 -arch x86_64' /usr/bin/python2.6 setup.py install

when using the Apple-supplied Python 2.6 or:

ARCHFLAGS='-arch i386' /usr/local/bin/pythonx.x setup.py install

when using a 32-bit-only python.org Python.

I think that distutils must me modified to auto detect supported architectures and use them, this without requiring you to hack ARCHFLAGS. 

The number of developers that need to build PPC binaries is at least one order of magnitude lower than the number of users in need of building/installing python packages.

All that said, the best solution to the problem is to use an appropriately configured Python for the task at hand.  If you want to build Python executables that are compatible with older OS X versions and Mac hardware (in particular, all machines capable of running OS X 10.4, 10.5, and 10.6), then you should be using something like the traditional python.org 32-bit-only installer configuration (i386/ppc, deployment target=10.3). And to build that Python or extension modules for it on OS X 10.6, you need to have Xcode 3 installed.  End of story.

However, if you don't need that kind of backward compatibility - either you're just running on your own 10.6 (or later) system or are only targeting 10.6+ systems, then you should be using something like the python.org 64-/32- installer configuration (x86_64/i386, deployment target=10.6).  For that you should be able to build Python or extension modules with either Xcode 4 or Xcode 3.

If you are targeting for 10.5 Intel-only and above and want to use Xcode 4, you would need a Python with a slightly different configuration (i.e. with a 10.5 deployment target); we don't currently supply an installer that meets that specialized (and presumably rapidly diminishing) need for x86_64 on 10.5 but it isn't that difficult to build yourself if you really need it.

So, if you choose the right binary installer for the task, there *shouldn't* be any issues with using Xcode 4.  If you need to build a Python from source, make sure you do not use the default ./configure values.  In particular, explicitly set --enable-universalsdk, --with-universal-archs, and MACOSX_DEPLOYMENT_TARGET.  For example,

  ./configure \
      --enable-universalsdk=/Developer/SDKs/MacOSX10.6.sdk \
      --with-universal-archs=intel \
      --enable-framework \
      MACOSX_DEPLOYMENT_TARGET=10.6

and any other options you might want like:
     --prefix=/path/to/bin
     --enable-framework=/path/to/framework
     --with-framework-name=whatever
         #which also influences the /Applications directory name

And, yes, longer term, we should do provide more appropriate defaults for these newer environments.  As we don't have a lot of experience yet with Xcode 4 and 10.7 has not been released yet, feedback is appreciated.

I don't agree that distutils should autodetect architectures, that can result in silent behavior changes when you built extensions and assume ppc support will get compiled in.

We should probably start to mention the intel only builds more prominently though, that's the one that will be useful to most people at this time.

Let's try to find a good compromise regarding this. I hope the we all agree that we should not put our personal needs or preferences on the first place, it's important to improve the overall experience for most users.

First, we do not want to fail to build and install a package if you are on specific case (xcode3, xcode4, ...).

I think that most people do not have `ARCHFLAGS` env var defined and they should not even care about it. Compilers are smart enough to pick the right architecture for them (xcode3 will build ppc/i386/x64 and xcode4 i386/x64, and who known maybe next version will include armX).

As for package maintainers and they build machines, they are more likely to be already aware about what they want to distribute. For this reason I consider that they can hack the build environment to meet their distribution needs (whatever env vars they want, blend xcode4 with xcode3,...). 

Still as stated in a previous comment, we do not want publish Universal binaries that will miss PPC architecture, in order to prevent this distutils should throw an warning if ppc architecture is not supported (xcode4).

Also if the ARCHFLAGS is defined and includes PPC, the build is supposed to fail.

Some other considerations: think that "some" people may want to install packages to their OS provided Python and in this case the python that comes with 10.6 doesn't have a PPC arch.

"Compilers are smart enough to pick the right architecture for them (xcode3 will build ppc/i386/x64 and xcode4 i386/x64"

Actually, that's not the case for Python builds.  The architecture selection is based on what the builder of the Python interpreter specified on the ./configure command and/or the defaults programmed into ./configure which look at other settings like deployment target. For extension module builds, Distutils attempts to use compile settings that are compatible with those the interpreter was built with, allowing some values to be influenced by environment variable settings.  In either case, the compiler has no say in the selection.

"Some other considerations: think that "some" people may want to install packages to their OS provided Python and in this case the python that comes with 10.6 doesn't have a PPC arch."

Actually, that's also not the case.  The Apple-supplied Pythons in OS X 10.6 do have a PPC arch; try "file /usr/bin/python2.6" in a shell and you'll see there are three archs.  It will even run on 10.6 in PPC emulation under Rosetta (try "arch -ppc /usr/bin/python2.6"),

But I sense a misconception here.  The python.org project has no say about what Apple does or doesn't do with regard to the system Pythons they supply in OS X. Python is an open-source project and many distributors (like Apple) build and release their own versions of Python. The PSF python.org project also builds and releases our own Windows and OS X binary installers.  But what is shipped with OS X is up to Apple.  For 10.6 the system python2.6 has a number of Apple-developed changes and add-ons.  The fact that Xcode 4 has complicated extension module builds with the system Python 2.6 is an issue to be taken up with Apple; there's nothing we here can do about that other than to suggest some workarounds.  And based on past experience, with the imminent release of 10.7, it's highly unlikely Apple will be making any non-security related changes to their Pythons in 10.6.  We'll have to see what 10.7 brings.  For the system Pythons on 10.6, the choice seems clear: either stick with Xcode 3 or resort to workarounds.

What we can do here is try to make things better for users of the python.org source or binary installers for currently supported versions of Python.  Documenting the tradeoffs of our current installer options is one step towards that.  Hope that makes things clearer!

You are right about Python from 10.6, it has PPC, but 10.7 doesn't.

bash-3.2$ file /Volumes/107/usr/bin/python2.6
/Volumes/107/usr/bin/python2.6: Mach-O universal binary with 2 architectures
/Volumes/107/usr/bin/python2.6 (for architecture x86_64):	Mach-O 64-bit executable x86_64
/Volumes/107/usr/bin/python2.6 (for architecture i386):	Mach-O executable i386

And in addition to 2.5 and 2.6, 10.7 includes 2.7.

What 10.7? ;-) Please remember 10.7 is under NDA, and as such shouldn't be discussed in public.

As I mentioned earlier: the 32-bit build should be deprecated and is only needed for older machines, anyone running modern machines can install OSX 10.6 (or later) and can use the intel-only build.

FYI, I got the above output from 10.7 from a friend, I do not have access to the seed, so the information could be wrong.

Closing this out of date issue.