2.6.5 release blocker
3.1.2 release blocker

For 10.6, new universal build options were added to reflect the dropping of support in 10.6 for the ppc64 arch.  The new options:

  --universal-archs=3-way -> -arch {i386,x86_64,ppc)
  --universal-archs=intel -> -arch {i386,x86_64}

however do not add the executables and symlinks to select a "-32" or
"-64" variant like the existing "--universal-archs=all" does and so
these new build variants always run in 64-bit with no way to override
it. A simple test in configure.in needs to be changed to recognize
these new options.

Trunk (2.7) and py3k (3.2) use a new execution-time mechanism to select 32- vs 64-bit execution and so do not exhibit this problem.

(Ronald and I agree this should be a release-blocker. Patch to follow.)

On Mar 08, 2010, at 08:20 PM, Ned Deily wrote:

>(Ronald and I agree this should be a release-blocker. Patch to follow.)

BTW, the right thing to do in this case (for 2.6 anyway) is to set the issue
to be a release blocker, so it gets my attention.  I can always knock it down
if I don't agree.

In this case I do, so I've set the priority for you.  Please try to fix this
asap so we can get rc2 out.

-Barry

The attached patches fix the problem for 2.6 and 3.1:
1. configure(.in) is changed to invoke the existing "4way" build targets for --with-universal-archs values of 3-way and intel as well as all.
2. configure(.in) passes the necessary lipo -extract arch values into the Mac/Makefile for building Python-32 and Python-64.  (Note, the solution here is simpler than what was required for trunk and py3k because the use of lipo is isolated to the "4way" build targets which can only be used for universal builds on 10.5+.)
3. a typo in the 2.6 Mac/README file is corrected and the previous updates are copied over from 2.6 to the 3.1 Mac/README.

Since configure.in is updated, it is necessary to run autoconf on both 2.6 and 3.1 after applying these patches.

Correct inadvertent deletion in the 3.1 Mac/README.

I've commited a fix for 2.6 in r78813.

I will port that fix to 3.1 later today, but don't have time to test right now.

My fix is simular to the patch by Ned.

Thanks for the fix guys.  I believe this means it's no longer a blocker for 2.6.5rc2, right?

Almost!  There's a small but significant typo that needs to be fixed in 
the change (r78813) that was committed for 2.6:

- UNIVERSAL_ARCH64_FLAGS="-arch x86_64 -arch x86_64"
+ UNIVERSAL_ARCH64_FLAGS="-arch x86_64 -arch ppc64"

Ned: I fixed the typo in r78816

Looks good for 2.6.5 (with 10.6 'intel' build).

Attempted to build 2.6.5rc2 on Mac Pro (2006), OS 10.6.2, following instructions in Mac/README:

./configure --prefix=/usr/local/tmp --enable-framework --with-universal-archs=intel --enable-universalsdk=/

Results of "make test" are as expected from past experience (but what architecture is being tested?):

332 tests OK.
2 tests failed:
    test_asynchat test_smtplib
32 tests skipped:
    test_al test_bsddb test_bsddb3 test_cd test_cl test_codecmaps_cn
    test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr
    test_codecmaps_tw test_curses test_dl test_epoll test_gl
    test_imageop test_imgfile test_largefile test_linuxaudiodev
    test_normalization test_ossaudiodev test_pep277 test_py3kwarn
    test_smtpnet test_socketserver test_startfile test_sunaudiodev
    test_timeout test_urllib2net test_urllibnet test_winreg
    test_winsound test_zipfile64
1 skip unexpected on darwin:
    test_dl

Notably, test_tcl passes without incident.

Running the following within the installation directory runs the 64-bit pre-installation executable (judging by sys.maxint):

$ DYLD_FRAMEWORK_PATH=`pwd`: python.exe

Thus I presume it is the 64-bit version being run by "make test".

The installed IDLE still has the Tk new window freeze issue; sys.maxint indicates it's the 32-bit version that is running when launcing IDLE.  I have a setup.py patch that will detect a 32-bit 10.6 installation and link against Apple's Tcl/Tk-8.4 compatibility version if the user hasn't
installed their own Tcl/Tk frameworks, but it's an ugly hack and I'm not sure this is desirable; we should probably just encourage installing newer Tcl/Tk.  The hack does produce a working 32-bit IDLE, however.

Running "python" on the command line runs the 32-bit version; is this intended?  I think the default behavior for universal builds should be documented in the Mac/README, or at some location pointed to in the README.

I still see no way to access the two architectures.  My ./configure installs command line executables in /usr/local/tmp/bin, a non-existing directory before the install.  After installation, its contents are:

idle@             pydoc2.6@         python2.6@        pythonw2.6@
idle2.6@          python@           python2.6-config@ smtpd.py@
pydoc@            python-config@    pythonw@          smtpd2.6.py@

I was expecting something along the lines of python-32 and/or python-64 to be here; what is the intended mechanism for accessing the two architectures?

$ arch -i386 python
gives me the (default) 32-bit version.

$ arch -x86_64 python
arch: posix_spawnp: python: Bad CPU type in executable

Whereas before with a universal install like this I was getting a 64-bit default and no way to access 32-bit (IIRC), now I appear to be in the reverse scenario.

Is there an error in my build process, or is there still an installation bug for dual-architecture Intel installs?

You need to run 'make install'.  The various "python-32" and "python-64" files are created and installed by the Mac/Makefile "install_python4way" target which should be getting called by the main Makefile "install" target as set by configure for the --with-universal-archs=intel.

As per msg100765, Ned confirms this is fixed for 2.6.5.

Re-opening for 3.1.2: the corresponding fixes have not made it into 3.1.2 yet.

Ned-

I *did* run "make install"; everything I reported was about the situation *after* running "make install".  In particular, I don't know any way to get access to IDLE without "make install"; what I described came from using the version in Applications/Python 2.6, which only appears after "make install".

Okay, now I see my misunderstanding---the arch-dependent executables are in the framework only, and not linked in the installation prefix.  Is that the intended behavior?  I do put my framework in my PATH so I do have access to the executables; but I was expecting them to be in /usr/local/... as well.

The version pointed to in /usr/local/... is the 32-bit version.  This should be documented somewhere; I believe it changes previous behavior.  It's not obvious to me what to expect here, which again argues that it should be documented.

On 15 Mar, 2010, at 16:22, Tom Loredo wrote:

> 
> Tom Loredo <loredo@astro.cornell.edu> added the comment:
> 
> Ned-
> 
> I *did* run "make install"; everything I reported was about the situation *after* running "make install".  In particular, I don't know any way to get access to IDLE without "make install"; what I described came from using the version in Applications/Python 2.6, which only appears after "make install".
> 
> Okay, now I see my misunderstanding---the arch-dependent executables are in the framework only, and not linked in the installation prefix.  Is that the intended behavior?  I do put my framework in my PATH so I do have access to the executables; but I was expecting them to be in /usr/local/... as well.
> 
> The version pointed to in /usr/local/... is the 32-bit version.  This should be documented somewhere; I believe it changes previous behavior.  It's not obvious to me what to expect here, which again argues that it should be documented.

This is not a regression in 2.6,  the python-32 executable has never been linked into /usr/local/bin.  I agree that they should be as long as we fill /usr/local/bin, but that can wait until 2.6.6.

I always add the framework to $PATH because distutils installs scripts into that location.   It would be worthwhile to change that for 2.7 and 3.2: tweak distutils to ensure that site-packages and scripts get installed in /Library/Python/2.7/... and install (links to) the public executables there as well.  That way the location of scripts and site-packages is easily found in the Finder, while keeping a versioned location.     I don't want to tweak distutils to install into /usr/local because I have several python trees and those should stay separate.

Ronald

> 
> ----------
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue8089>
> _______________________________________

> the python-32 executable has never been linked into /usr/local/bin.

What I meant by "the version pointed to" is:  The "python" command
in 2.6.4 produced by an "intel" universal build (whether in the
framework or the install prefix) executed as 64-bit.  I
don't believe it linked to either python-32 or -64; I don't think those
were even installed in 2.6.4.  I think it was a bundle that just
happened to execute as 64-bit (and provided no access to 32-bit
python).  So the "python" command-line behavior under 2.6.5rc2 is
different from what it was for 2.6.4.  I don't think this is a 
significant regression, as the previous behavior was never documented 
and was actually somewhat problematic.  I'm just pointing it out for 
documentation purposes.

> I always add the framework to $PATH because distutils installs scripts into that location.

Yes, I do the same.  Still, I think your proposed plans for linking in the prefix in 2.7/3.2 make a lot of sense.

Unless you vehemently disagree, I am not making this a release blocker for 2.6.5.

> Unless you vehemently disagree, I am not making this a release blocker for 2.6.5.

I'm not sure who you are asking (I doubt it was me!), but I don't consider this a release blocker.  The only possible substantive issue is whether "python" should run 32-bit or 64-bit Python after a universal framework install.  Previously it ran 64-bit, but that was probably accidental.  I don't have a strong opinion about this; others should weigh in appropriately.

I think a change in behavior here would only negatively impact a very small number of users (those who built from source on OS X, and who built universal versions on a 64-bit platform, and who count on 64-bit default behavior).  For example, they may have built 64-bit extensions that could appear broken after an update to 2.6.5 if they don't know they should use "python-64".

In my opinion the main part of the "source" at issue here is not really source but the Mac/README file; but this is certainly not the only subtlety of Python-on-OS X that is not yet covered there!

Per discussion with Ronald and Benjamin, attaching a revised patch (issue-sl-configure-32-31-rev3.txt) for 3.1 which ports the changes committed in r78813 and r78816 along with the Mac/README changes.

In the process of producing the revised 3.1 patch, I discovered that the 2.6 changes as committed produce a build error for the 10.5 --with-universal-archs=all (4-way) variant.  The 3.1 revised patch corrects that error.  Since 2.6.5 is now frozen, I am opening a new issue with patch for 2.6.5.

I recommend that, once the 3.1 patch is applied for 3.1.2, this issue be closed and any new or remaining issues be tracked in new or other existing issues.

(2.6.5 -> Issue8175)

Fixed 3.1 in r79117.