Title: Distutils inappropriately reuses .o files between extension modules
Type: behavior Stage: resolved
Components: Distutils Versions: Python 3.1, Python 3.2, Python 2.7, Python 2.6
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: tarek Nosy List: collinwinter, eric.araujo, jyasskin, skip.montanaro, tarek
Priority: normal Keywords: needs review, patch

Created on 2009-02-26 00:05 by collinwinter, last changed 2012-05-04 03:16 by eric.araujo. This issue is now closed.

File name Uploaded Description Edit
ccompiler.patch collinwinter, 2009-02-26 00:05 Patch against trunk r69982
Messages (5)
msg82734 - (view) Author: Collin Winter (collinwinter) * (Python committer) Date: 2009-02-26 00:05
(Tarek, I've been told you're the new distutils maintainer. Feel free to
unassign this if that isn't the case.)

The test distutils uses to decide whether it needs to recompile an
existing .o file when building extension modules is too simplistic,
merely comparing the modification time of the .o and .c files. If you
have two extension modules like so

Extension('foo', ['foo.c', 'bar.c'],
          define_macros=[('NO_STATIC_MEMOTABLE', 1)])

Extension('bar', ['bar.c'])

even if defining NO_STATIC_MEMOTABLE=1 radically changes the code of
bar.c, distutils will use the same bar.o for both the foo and bar
extension modules. This was a real problem for me at work today.

The attached patch removes the modtime comparison entirely, preferring
to rebuild all .o files a given extension module needs. Note that while
_prep_compiler() isn't used anywhere in the stdlib, Google Code Search
turns up uses.
msg97493 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2010-01-10 02:38
LGTM.  Nothing is quite as satisfying as simply deleting a bunch of
msg97556 - (view) Author: Tarek Ziadé (tarek) * (Python committer) Date: 2010-01-10 22:38
This optimization is indeed simplistic. 

I'll also remove _prep_compiler() in trunk, as I don't see a problem in removing it in the 2.7.x and 3.2.x series.

I also need to fix it in 2.6.x / 3.1.x, but I'll keep the private method there.
msg97614 - (view) Author: Tarek Ziadé (tarek) * (Python committer) Date: 2010-01-11 23:24
done in r77424, r77425, r77427, r77430

Thanks Collin !
msg159895 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-05-04 03:16
See also #14517.
Date User Action Args
2012-05-04 03:16:33eric.araujosetnosy: + eric.araujo
messages: + msg159895

resolution: accepted -> fixed
stage: patch review -> resolved
2010-01-11 23:24:06tareksetstatus: open -> closed
keywords: patch, patch, needs review
messages: + msg97614
2010-01-10 22:38:04tareksetpriority: normal
versions: + Python 2.6, Python 3.1, Python 3.2
messages: + msg97556

keywords: patch, patch, needs review
resolution: accepted
2010-01-10 02:38:41skip.montanarosetkeywords: patch, patch, needs review
nosy: + skip.montanaro
messages: + msg97493

2009-02-26 00:05:26collinwintercreate