Title: Broken makefile depends for profile-opt target
Type: enhancement Stage: resolved
Components: Build Versions:
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, dgreiman, gregory.p.smith, nascheme, vstinner
Priority: normal Keywords: patch

Created on 2016-11-08 19:56 by nascheme, last changed 2018-07-16 17:56 by nascheme. This issue is now closed.

File name Uploaded Description Edit
profile-opt-stamp.txt nascheme, 2016-11-08 21:44
Pull Requests
URL Status Linked Edit
PR 4223 merged nascheme, 2017-11-01 21:09
Messages (6)
msg280342 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2016-11-08 19:56
I notice that after running "make" then running "make install", the build will go through the whole compile/profile/compile process again.  This is really infuriating behaviour, given the extremely long make time for the profiled optimized build.

The problem appears to me to be that the "profile-opt" target does not have proper dependencies.  I think changing it to:

profile-opt: $(BUILDPYTHON)

should fix it.  If my "make install" ever finishes, maybe I will test it. ;-)
msg280354 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2016-11-08 21:44
Okay, my initial idea was wrong (I blame years of not having to look at Makefiles).  I think the attached patch works.  It uses a "stamp" file to record the fact that the profiled build is complete.

The fix is sub-optimal because changing some source code and re-running "make" will not rebuild as needed.  That would require proper dependencies in the Makefile, rather than treating "make" as an imperative scripting language (e.g. recursively calling make to sequence things).  Given that the profile optimised build is unlikely to be used during development, I guess that's not so bad.
msg305443 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2017-11-02 17:46
New changeset 4e38d71a2b7b606fb1b0078c2d7014fc24044dac by Neil Schemenauer in branch 'master':
bpo-28643: Record profile-opt build progress with stamp files (#4223)
msg305661 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-11-06 17:20
Oh, I had this bug and I'm quite sure that I reported it, but I cannot find it anymore :-)
msg305664 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2017-11-06 17:38
The previous behavior nearly drove me to drink.  At least on my machine (and I have a relatively fast one), the profile-opt build takes a long time.  After running "make" and checking things over, running "make install" will cause the whole process (make clean, make with -fprofile-generate, run unit tests, make clean, make with -fprofile-use) to happen again.  The profile-opt build is significantly faster so I like to use it.  Maybe I'm an odd duck in that I usually compile Python myself rather than using distro packaged versions.

Current behavior is much better I think.  You have to know to manually remove "profile-run-stamp" if you want the -fprofile-generate + unittest to run again.  "make clean" does not remove it.  I don't know if that should be documented somewhere besides in the Makefile.
msg321750 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2018-07-16 17:56
Closing this as the current makefile works much better in this regard, at least based on my experience.
Date User Action Args
2018-07-16 17:56:54naschemesetstatus: open -> closed
resolution: fixed
messages: + msg321750

stage: patch review -> resolved
2017-11-06 17:38:43naschemesetmessages: + msg305664
2017-11-06 17:20:06vstinnersetnosy: + vstinner
messages: + msg305661
2017-11-02 17:46:04naschemesetmessages: + msg305443
2017-11-01 21:09:32naschemesetkeywords: + patch
pull_requests: + pull_request4191
2017-04-07 21:02:57dgreimansetnosy: + dgreiman
2016-11-09 17:01:00brett.cannonsetstage: needs patch -> patch review
2016-11-09 17:00:52brett.cannonsetnosy: + brett.cannon, gregory.p.smith
2016-11-08 21:44:44naschemesetfiles: + profile-opt-stamp.txt

messages: + msg280354
2016-11-08 19:56:35naschemecreate