classification
Title: Separate build dir broken
Type: compile error Stage: resolved
Components: Build Versions: Python 3.2, Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: mwh Nosy List: andybuckley, doko, eric.araujo, mwh, nascheme, rpetrov, tarek
Priority: normal Keywords: patch

Created on 2008-10-20 17:04 by nas, last changed 2010-09-20 22:02 by eric.araujo. This issue is now closed.

Files
File name Uploaded Description Edit
get_python_inc2.patch nas, 2008-10-20 17:06 3.0+ patch
get_python_inc2.patch nas, 2008-10-20 17:06 2.6+ patch
non_posix_srcdir.patch nas, 2008-10-22 16:13
Messages (15)
msg74996 - (view) Author: Neil Schemenauer (nas) Date: 2008-10-20 17:04
Building in a separate directory got broken at some point.  The code is
hairy but it looks like the source of the problem was a lame
sysconfig.get_python_inc() function.  The attached patches fix things
and hopefully do not introduce any new bugs.
msg75013 - (view) Author: Roumen Petrov (rpetrov) * Date: 2008-10-20 22:21
get_python_inc() and test case is addressed in issue4070 as well.
msg75085 - (view) Author: Neil Schemenauer (nas) Date: 2008-10-22 16:13
[Roumen Petrov]
> Modification for test_build_ext.py from above mentioned issue break
> non-posix builds.

Without that change the test fails on Unix platforms when the build is
not done inside the source directory.  I guess the problem must be the
change from (essentially):

def _get_source_filename():
    return os.path.join(sysconfig.project_base, 'Modules', 'xxmodule.c')

to

def _get_source_filename():
    srcdir = sysconfig.get_config_var('srcdir')
    return os.path.join(srcdir, 'Modules', 'xxmodule.c')

On POSIX, project_base is just the directory containing the Python
executable.  I think the right thing to do is to fix the srcdir var on
non-POSIX systems.  I don't have a Windows machine to test on.  Could
someone test the get_python_inc2.patch with the non_posix_srcdir.patch
applied on top of it?
msg81231 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2009-02-05 21:17
I committed my proposed changes in several chunks, ending with r69305. 
I think building in a separate directory again works and that non-POSIX
platforms are not adversely affected by this change.
msg81232 - (view) Author: Roumen Petrov (rpetrov) * Date: 2009-02-05 22:30
May be not related to the commit but after clean make fail to build a
number of modules, as example _bisect. Second run of make build all left.

The value of srcdir (from Makefile) is ".." without quotes.


About cleanup: one use of '(srcdir,)==...' left in setup.py .
msg81234 - (view) Author: Roumen Petrov (rpetrov) * Date: 2009-02-05 22:41
The "clean build' mean missing subdirectory build in <builddir> . Now
module objects go in build/Modules/. I guess that second make succeed as
libffi create build directory.
msg81242 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2009-02-06 00:38
Roumen, thanks for reporting the bug.  It seems that distutils requires
an absolute path (although I didn't waste time digging too deep into the
issue).
msg81763 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2009-02-12 13:32
still seen on the 2.6 branch. applying r69374 on the branch doesn't fix
it there.
msg81796 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2009-02-12 18:29
On Thu, Feb 12, 2009 at 01:32:37PM +0000, Matthias Klose wrote:
> still seen on the 2.6 branch. applying r69374 on the branch doesn't fix
> it there.

The fix is spread over a number of commits: r69374, r69322, r69315,
r69305, r69304, r69303, r69302.  It could be backported but changing
the build system is always fraught with portability traps and that's
why it ends up turning to crud over time.

Obviously most core Python developers don't build in a separate
directory otherwise this feature won't have been broken for so long.
msg82247 - (view) Author: Andy Buckley (andybuckley) Date: 2009-02-16 16:19
I'm having trouble with this as well :(  It's pretty much a blocker for
integrating distutils-based extension builds with an autotools library
build, because the "make distcheck" target explicitly does the build in
a subtree of the source directory, and write-protects the srcdir during
the build.

Effectively the source is referred to in my setup.py as living in
"../../pyext", which translates into distutils attempting to build the
extension in "build/temp.linux-i686-2.5/../../pyext/", which is the
write-protected source dir. This would work okay if the directory path
to the source file were ignored when constructing the output path: would
the proposed patch(es) fix this?
msg82265 - (view) Author: Roumen Petrov (rpetrov) * Date: 2009-02-16 19:52
One additional commit change  srcdir = os.path.normpath(srcdir) to
srcdir = os.path.abspath(srcdir) and this should not create relative
path. In this case I expect build in directory
build/temp.<PLATFORM_VERSION>/<ABS_SRC_PATH>.

Andi, did above change work for you.
msg99402 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2010-02-16 12:52
current status with 2.7 alpha3:

FAIL: test_get_python_inc (distutils.tests.test_sysconfig.SysconfigTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/packages/python/2.7/python2.7-2.7~a3/Lib/distutils/tests/test_sysconfig.py", line 47, in test_get_python_inc
    self.assertTrue(os.path.isfile(python_h), python_h)
AssertionError: /home/packages/python/2.7/python2.7-2.7~a3/build-static/Include/Python.h

unsure about this one:
FAIL: test_finalize_options (distutils.tests.test_build.BuildTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/packages/python/2.7/python2.7-2.7~a3/Lib/distutils/tests/test_build.py", line 24, in test_finalize_options
    self.assertEquals(cmd.build_purelib, wanted)
AssertionError: 'build/lib.linux-i686-2.7' != 'build/lib'
msg108876 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2010-06-28 22:49
I believe the latest problem reported has been fixed by SVN rev 80649.
msg115135 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-08-27 22:01
I think so too.
msg116987 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-09-20 22:02
For the record, I’m not clear about what is fixed or not in each branch.
History
Date User Action Args
2010-09-20 22:02:11eric.araujosetmessages: + msg116987
2010-08-27 22:01:28eric.araujosetstatus: open -> closed

nosy: + mwh
messages: + msg115135

assignee: nascheme -> mwh
resolution: fixed
2010-06-28 22:49:40naschemesetnosy: - nas
messages: + msg108876

assignee: nascheme
stage: resolved
2010-04-09 01:40:49eric.araujosetnosy: + eric.araujo
2010-02-16 12:52:39dokosetnosy: + tarek

messages: + msg99402
versions: + Python 2.7, Python 3.2
2009-02-16 19:52:35rpetrovsetmessages: + msg82265
2009-02-16 16:19:37andybuckleysetnosy: + andybuckley
messages: + msg82247
2009-02-12 18:29:29naschemesetmessages: + msg81796
2009-02-12 13:32:36dokosetstatus: closed -> open
nosy: + doko
resolution: fixed -> (no value)
messages: + msg81763
versions: + Python 2.6
2009-02-06 00:38:50naschemesetmessages: + msg81242
2009-02-05 22:41:02rpetrovsetmessages: + msg81234
2009-02-05 22:30:24rpetrovsetmessages: + msg81232
2009-02-05 21:17:53naschemesetmessages: - msg81230
2009-02-05 21:17:35naschemesetstatus: open -> closed
resolution: fixed
messages: + msg81231
nosy: + nascheme
2009-02-05 21:14:46nassetmessages: + msg81230
2008-10-22 16:13:39nassetfiles: + non_posix_srcdir.patch
messages: + msg75085
2008-10-20 22:21:44rpetrovsetnosy: + rpetrov
messages: + msg75013
2008-10-20 17:06:53nassetfiles: + get_python_inc2.patch
2008-10-20 17:06:33nassetfiles: - get_python_inc.patch
2008-10-20 17:06:28nassetfiles: + get_python_inc2.patch
2008-10-20 17:04:15nascreate