Author ned.deily
Recipients alexis, benjamin.peterson, eric.araujo, georg.brandl, kontinuity, ned.deily, python-dev, ronaldoussoren, srid, stoneyb, tarek
Date 2011-06-13.21:13:42
SpamBayes Score 0.0
Marked as misclassified No
Message-id <>
There are several issues here now.

With the patches as now applied, when running the tests with standard OS X installer Pythons, I saw occasional failures of the new test_deployment_target test case ("Unexpected target").   After ensuring that the build_ext tests actually are executed consistently (see the patches for Issue12141), I found that the test case failed consistently when test_distutils was run after test___all__, which is the default.  There was also a telltale warning that test___all__ altered the execution environment.  After investigating, I've come to the conclusion that the root cause of this test failure *and* of the original reported problem in this issue is the original implementation of forcing MACOSX_DEPLOYMENT_TARGET way back in r38123.

The main problem with r38123 is that, while the issue needing fixing is limited to Distutils-spawned build steps, the solution unpredictably can modify the environment of *all* subprocesses (as Ronald noted earlier in msg114195):

>>> import os
>>> os.system('echo $MACOSX_DEPLOYMENT_TARGET')

>>> import
>>> os.system('echo $MACOSX_DEPLOYMENT_TARGET')

The modification occurs in distutils.sysconfig._init_posix when distutils.sysconfig is imported so the env change can be a side-effect of importing other distutils modules, like above.  This behavior has been the case since r38123 but its effects were somewhat masked within the Python interpreter process by the use of os.putenv.  Now that the patches for this issue replaced os.putenv with setting os.environ directly, the changed MACOSX_DEPLOYMENT_TARGET is now visible in the interpreter, too.  And that's the cause of the new test___all__ "altered the execution environment" message.  Running test___all__ causes distutils.sysconfig to be imported which causes the _init_posix one-time initialization to take place which will set MACOSX_DEPLOYMENT_TARGET (if it wasn't already set externally).  Regrtest now sees the change after running test___all__ and restores the original environment thereby deleting MACOSX_DEPLOYMENT_TARGET.  Because _init_posix is only run once, test_distutils fails afterwards because MACOSX_DEPLOYMENT_TARGET is no longer set and will never be set.

So I am now convinced that, indeed, the right solution is to only set MACOSX_DEPLOYMENT_TARGET in the Distutils-spawned processes.  Setting it globally is too fragile and has unintended side-effects (like in the reported problems earlier in this issue).

Beyond that there are some issues with the new test cases.  As it stands, the new test_deployment_target test case tries to run two subtests but the second doesn't actually work: distutils.build_ext doesn't bother recompiling it since the object file leftover from the first subtest appears to be up-to-date.  Also the test cases do not fully test the intended behavior of the deployment target processing:  allowing the deployment target to be overridden to be equal or greater than that of the interpreter build but not less.

Another issue is that with packaging now landed in the default branch (for 3.3), similar code and tests are needed there.  √Čric ported the new test cases but distutils.sysconfig does not exist in packaging: it uses the standard sysconfig.  So there is currently no deployment target checking or setting (what's done in distutils.sysconfig._init_posix) at all when using packaging.

The attached patches for 3.3, 3.2, and 2.7 try to address all these issues:
 - The deployment target setting code is removed from
 - Distutils.spawn is modified to set the appropriate deployment
   target in the environment of spawned build subprocesses.
 - In test_build_ext, test case test_deployment_target is replaced
   by three new test cases: test_deployment_target_default,
   and test_deployment_target_higher_ok.
 - For 3.3 only, similar code is added to spawn in packaging.util
   and the test cases added to test_command_built_ext.
Date User Action Args
2011-06-13 21:13:45ned.deilysetrecipients: + ned.deily, georg.brandl, ronaldoussoren, benjamin.peterson, tarek, eric.araujo, srid, alexis, stoneyb, kontinuity, python-dev
2011-06-13 21:13:45ned.deilysetmessageid: <>
2011-06-13 21:13:44ned.deilylinkissue9516 messages
2011-06-13 21:13:43ned.deilycreate