classification
Title: distutils does not deduce dependencies
Type: enhancement Stage:
Components: Distutils Versions:
process
Status: closed Resolution: later
Dependencies: Superseder:
Assigned To: Nosy List: akuchling, gvanrossum, mwh, skip.montanaro
Priority: normal Keywords:

Created on 2001-10-19 18:49 by skip.montanaro, last changed 2003-02-06 18:03 by akuchling. This issue is now closed.

Files
File name Uploaded Description Edit
setup.diff mwh, 2001-12-10 12:39 Not The Smartest Fix In The World
Messages (11)
msg7066 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2001-10-19 18:49
I just "cvs up"d my Python tree and executed "make".
Nothing much changed, except patchlevel.h.  Make got
this right and rebuilt everything (patchlevel.h is
included from Python.h, so changing patchlevel.h
should cause make to rebuild just about everything).

Distutils failed to notice this, however, so it didn't rebuild any modules.  In this case, I think the
failure to rebuild is probably innocuous, but I'm
not at all confident it will do the right thing if I
modify some other file.  For example, I've noticed
that it's not sufficient to delete a module's .o file
when you want to rebuild it.  Distutils still thinks
the .so file is okay.

If distutils is going to take the place of make I
think it's going to need to do a much better job
deducing file dependencies or provide a robust
way for a person to tell it what those dependencies
are.
msg7067 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2001-12-07 12:36
Logged In: YES 
user_id=6656

Would a "good enough" solution be to check (in setup.py) if
pyconfig.h or anything in Include/ has changed since the
last build and if so blow away the & build directory?

Doing a "proper" job seems frankly unlikely, and would have
much the same effect as the above anyway for building Python...
msg7068 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2001-12-07 13:40
Logged In: YES 
user_id=6380

Hm, how do you know when the last build was?

Also, since in 99% of the cases this is unnecessary and
rebuilding all extensions takes a long time, I'd like this
to be optional -- maybe it can ask the user before zapping
build?
msg7069 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2001-12-07 13:53
Logged In: YES 
user_id=6656

Eh, I was going to add a os.system("touch last_build")
thingy to setup.py.  Not aiming for elegance here.

Is it really 99% of the time that changes in headers are
irrelevant?  Then what make does is a bit silly, surely.

It would presumably be easy enough to ask the user, but I
know I'd find that really annoying (I generally fire a biuld
off, do something else for a few minutes and come back. 
OTOH, I almost always go through a "rm -rfv build && mkdir
build && cd build && ../configure --prefix=$HOME && make"
ritual anyway so this issue doesn't affect me in the slightest).
msg7070 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2001-12-07 14:54
Logged In: YES 
user_id=6380

It depends on what kind of change you are making to a header
file. 99% of the time I find that a change to a header is
something innocuous like add a new function, and it still
causes a rebuild of the world. That's particularly annoying
when I'm experimenting with a new object type and adding
things to the header file one at a time -- each time the
header gets touched everything gets rebuilt.

But it's easy enough to only rebuild the core ("make
python") so I'm not sure if I really oppose your simple
solution.
msg7071 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2001-12-07 15:49
Logged In: YES 
user_id=6656

OK, I'll try to get something concrete done in the next few
days.
msg7072 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2001-12-10 12:39
Logged In: YES 
user_id=6656

Here's an attempt.  TBH, I'd be a bit nervous about fiddling
with the build at this stage, but assigning to Guido to decide.

I'd recommend assigning it back to me Pending/Later, and
applying something like this early in 2.3.
msg7073 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2001-12-10 15:16
Logged In: YES 
user_id=6380

I agree.  Let's put this off until 2.3 (or maybe 2.2.1).
msg7074 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2002-01-23 10:28
Logged In: YES 
user_id=6656

Umm, I didn't mean to close this.  Oh well.

TBH, I'm tempted to close it again, with a resolution of
"wont fix", but am open to persuasion.

I'm not sure I can summon the motivation to add dependency
tracking to distutils.  And I don't like the hack I've
attached to the report.
msg7075 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2002-12-05 18:20
Logged In: YES 
user_id=6656

This has been sitting on my plate forever.

AIUI, distutils can now be told about dependencies, but does
nothing to deduce them itself.  As this puts it in good
company (eg, make), I reckon we can probably close this?

At any rate, I don't see me getting to it.
msg7076 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2003-02-06 18:03
Logged In: YES 
user_id=11375

Added to PEP 42 and marking as closed.
History
Date User Action Args
2001-10-19 18:49:41skip.montanarocreate