classification
Title: distutils.util.get_platform() is wrong for universal builds on macosx
Type: Stage:
Components: Distutils Versions: Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: mw263
Priority: critical Keywords: needs review, patch

Created on 2008-10-07 09:41 by ronaldoussoren, last changed 2009-10-11 17:41 by ned.deily. This issue is now closed.

Files
File name Uploaded Description Edit
issue4064-fix.patch ronaldoussoren, 2008-10-07 09:42
Messages (15)
msg74432 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2008-10-07 09:41
When using a universal build of python on macosx 
distutils.util.get_platform should use "fat" for the machine 
architecture, instead of the architecture of the current machine.

That's not what's currently happening:

$ python26 -c 'from distutils.util import get_platform; print 
get_platform()'
macosx-10.3-ppc
$
msg74433 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2008-10-07 09:42
The attached patch fixes this issue.
msg74468 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-10-07 19:29
> When using a universal build of python on macosx 
> distutils.util.get_platform should use "fat" for the machine 
> architecture, instead of the architecture of the current machine.

Can you please explain why it should do so? Where do these architecture
names come from?
msg74473 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2008-10-07 19:52
The basic idea is that the architecture bit of get_platform() should 
tell you something about the archicture for which a build is valid. 
That's why 'i386' or 'ppc' is not very useful for a universal build. The 
original author of the universal binary support choose 'fat' as the 
architecture designator for universal builds ("fat binaries").

Somewhere along the way the calculation of the architecture string got 
messed up, resulting in the current situation. That is, the current 
situation is not as designed by the original author of the universal 
binary support code.

Another reason for fixing this is that setuptools uses the platform 
designator of the current system and a downloaded binary archive to 
determine if the two are compatible. In the current situation univeral 
binary distributions built on a PPC system don't work on an i386 system 
(as far as setuptools is concerned).
msg74477 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-10-07 20:13
> Somewhere along the way the calculation of the architecture string got 
> messed up, resulting in the current situation. That is, the current 
> situation is not as designed by the original author of the universal 
> binary support code.

Ok, what *is* the desired way of naming architectures on that system?
Is that documented somewhere?
msg74514 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2008-10-08 06:02
On 7 Oct, 2008, at 22:13, Martin v. Löwis wrote:

>
> Martin v. Löwis <martin@v.loewis.de> added the comment:
>
>> Somewhere along the way the calculation of the architecture string  
>> got
>> messed up, resulting in the current situation. That is, the current
>> situation is not as designed by the original author of the universal
>> binary support code.
>
> Ok, what *is* the desired way of naming architectures on that system?
> Is that documented somewhere?

It is not documented anywhere but in the code, but this is the  
intended behaviour:

* A single architecture build (what you would build on any other unix  
platform)
   uses the architecture string of the architecture
   it was build for (that is, "i386", "ppc", "x86_64" or "ppc64")

* A 32-bit universal build uses "fat" as the architecture string,  
regardless of the
    architecture of the machine we're running on.
    As an exception: OS X 10.3.9 machines are treated as if we're on a  
single
    architecture build because the compiler on that platform doesn't  
support
    building fat binaries.

* A 4-way universal build (that is, including all architectures  
supported by
    OSX) uses "universal" as the architecture string.

We (Bob Ippolitto and I) had some discussion about the architecture  
strings when
we were working on support for universal binaries and rejected my  
initial suggestion
of using "i386,ppc" instead of "fat" because that would be unwieldy.

Ronald
msg74515 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-10-08 06:55
> It is not documented anywhere but in the code

These also appear in file names of bdist commands, right? So I think it
should be documented.

> We (Bob Ippolitto and I) had some discussion about the architecture  
> strings when
> we were working on support for universal binaries and rejected my  
> initial suggestion
> of using "i386,ppc" instead of "fat" because that would be unwieldy.

OK. I wonder how you will call fat 64-bit binaries (i.e. ppc64 and
amd64), but I can live with that semantics as long as it's documented
(I actually question that it is documented in the code. If somebody
would put "-arch Itanium" in her CFLAGS, which might be supported in
10.9, it would infer that the architecture is "fat").
msg78428 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2008-12-28 19:52
I've committed the patch with some documentation in r67988, with a 
backport to 2.6.x in r67989.  r67990 (not backported) is a minor update
of the patch, it adds explicit support code for all three variants that
are configurable through the configure script.

I don't think it is useful to worry about architectures that are not 
supported by current releases of MacOSX.
msg93735 - (view) Author: Michael Wise (mw263) Date: 2009-10-08 08:41
While the discussion seems to think the matter is closed, I wanted to
install numpy 1.30 which requires Python 2.6 and all my machines are PPC
(G4 or G5 - nice architectures). 

gcc-4.0 -arch ppc -arch i386 -fno-strict-aliasing ...

followed, unsurprisingly, by:
gcc-4.0: installation problem, cannot exec
'i686-apple-darwin8-gcc-4.0.0': No such file or directory

Work around, please?

Cheers
MichaelW
msg93736 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-10-08 08:56
Michael: please file a new issue for this, your problem seems to be 
unrelated to this one.

In that issue include information about:

1) The python version you are using
2) The version of MacOSX
3) The version of Xcode (open /Developer/Applications/Xcode.app,
    check the version in the About dialog)
msg93823 - (view) Author: Michael Wise (mw263) Date: 2009-10-10 08:06
Dear Ronald

Not that simple. I had, for the first time, installed Python 2.6.3 via 
the .dmg rather than compiled from scratch, and then numpy, again via 
the .dmg. I was trying to compile biopython from scratch using disutils 
when the problem occurred. That was on a G4 PPC Mac laptop. On a 
different G5 PPC  PowerMac, I did the whole job from scratch and all was 
well.

I can confirm that get_platform was not the problem (check it 
specifically) and I got something to work by modifying the 
config/Makefile, but then worried about what else that would affect. I 
removed the .dmg supplied Python and numpy, and installed Python 2.6.3 
from scratch, but had problem of the disutils installation of numpy 
hanging, so I removed that and did the whole job based on the previous 
installation 2.5.2.

Cheers
MichaelW

Ronald Oussoren wrote:
> Ronald Oussoren <ronaldoussoren@mac.com> added the comment:
> 
> Michael: please file a new issue for this, your problem seems to be 
> unrelated to this one.
> 
> In that issue include information about:
> 
> 1) The python version you are using
> 2) The version of MacOSX
> 3) The version of Xcode (open /Developer/Applications/Xcode.app,
>     check the version in the About dialog)
> 
> ----------
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue4064>
> _______________________________________
>
msg93824 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2009-10-10 09:24
>gcc-4.0 -arch ppc -arch i386 -fno-strict-aliasing ...
>
>followed, unsurprisingly, by:
>gcc-4.0: installation problem, cannot exec
>'i686-apple-darwin8-gcc-4.0.0': No such file or directory

From at least OS X 10.4 on, Xcode installs both variants of compilers so 
it is possible to build both Intel and PPC archs on any system.  I 
regularly build complete fat Python installers on a PPC G3 running 10.4.

$ uname -p
powerpc
$ i686-apple-darwin8-gcc-4.0.1 
i686-apple-darwin8-gcc-4.0.1: no input files

Was this possibly on an older version of OS X, say 10.3?  Otherwise, it 
sounds like that system did not have a complete installation of Xcode 
somehow.
msg93826 - (view) Author: Michael Wise (mw263) Date: 2009-10-10 11:54
Dear Ned

Odd you should say that. The system is the latest version of 10.4 
(10.4.11),  but I did notice that the compiler assumed 10.3. The version 
  of Xcode on this PowerBook G4 is 2.0 (quite old), so perhaps it has 
OSX 10.3 wired in. I don't think that was the issue, but to eliminate it 
I'll download Xcode when I'm at work on Monday (better bandwidth) and 
see if that does a better job.

Cheers (and thanks!)
Michael

Ned Deily wrote:
> Ned Deily <nad@acm.org> added the comment:
> 
>> gcc-4.0 -arch ppc -arch i386 -fno-strict-aliasing ...
>>
>> followed, unsurprisingly, by:
>> gcc-4.0: installation problem, cannot exec
>> 'i686-apple-darwin8-gcc-4.0.0': No such file or directory
> 
>>From at least OS X 10.4 on, Xcode installs both variants of compilers so 
> it is possible to build both Intel and PPC archs on any system.  I 
> regularly build complete fat Python installers on a PPC G3 running 10.4.
> 
> $ uname -p
> powerpc
> $ i686-apple-darwin8-gcc-4.0.1 
> i686-apple-darwin8-gcc-4.0.1: no input files
> 
> Was this possibly on an older version of OS X, say 10.3?  Otherwise, it 
> sounds like that system did not have a complete installation of Xcode 
> somehow.
> 
> ----------
> nosy: +ned.deily
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue4064>
> _______________________________________
>
msg93831 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-10-10 15:53
Michael: Again, please file a new issue for your problem because it is not 
related to this one.

I'm removing myself from the nosy-list for this bug.
msg93864 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2009-10-11 17:41
10.3 is a red herring, that is the deployment target and is as expected. 
If you install the current proper versions of Xcode for your 10.4 or later 
systems, things will work as designed.  Again, this is not the proper 
forum to discuss build problems.  If you still have problems, suggest 
asking questions in the pythonmac-sig mailing list.
History
Date User Action Args
2009-10-11 17:41:59ned.deilysetnosy: - ned.deily
2009-10-11 17:41:43ned.deilysetnosy: ned.deily, mw263
messages: + msg93864
2009-10-11 10:56:05ronaldoussorensetnosy: - ronaldoussoren
2009-10-11 05:52:12loewissetnosy: - loewis
2009-10-10 15:53:31ronaldoussorensetmessages: + msg93831
2009-10-10 11:54:36mw263setmessages: + msg93826
2009-10-10 09:24:34ned.deilysetnosy: + ned.deily
messages: + msg93824
2009-10-10 08:06:41mw263setmessages: + msg93823
2009-10-08 08:56:13ronaldoussorensetmessages: + msg93736
2009-10-08 08:41:09mw263setnosy: + mw263
messages: + msg93735
2008-12-28 19:52:39ronaldoussorensetstatus: open -> closed
resolution: fixed
messages: + msg78428
2008-10-08 06:55:33loewissetmessages: + msg74515
2008-10-08 06:02:14ronaldoussorensetmessages: + msg74514
2008-10-07 20:13:29loewissetmessages: + msg74477
title: distutils.util.get_platform() is wrong for universal builds on macosx -> distutils.util.get_platform() is wrong for universal builds on macosx
2008-10-07 19:52:56ronaldoussorensetmessages: + msg74473
2008-10-07 19:29:20loewissetnosy: + loewis
messages: + msg74468
title: distutils.util.get_platform() is wrong for universal builds on macosx -> distutils.util.get_platform() is wrong for universal builds on macosx
2008-10-07 12:08:03ronaldoussorensetkeywords: + needs review
2008-10-07 09:43:22ronaldoussorensetpriority: critical
2008-10-07 09:42:59ronaldoussorensetfiles: + issue4064-fix.patch
keywords: + patch
messages: + msg74433
2008-10-07 09:41:45ronaldoussorencreate