classification
Title: `make regen-all` triggers wholesale rebuild
Type: enhancement Stage: resolved
Components: Build Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: pitrou, serhiy.storchaka, twouters, vstinner, zach.ware
Priority: normal Keywords: patch

Created on 2017-09-20 18:37 by pitrou, last changed 2017-09-28 11:21 by pitrou. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 3677 closed pitrou, 2017-09-20 18:39
PR 3678 merged pitrou, 2017-09-20 19:28
PR 3797 merged python-dev, 2017-09-28 10:41
Messages (12)
msg302649 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-09-20 18:37
`make regen-opcode` always writes the destination unconditionally, after which `make` thinks it has to rebuild everything.
msg302651 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-09-20 18:42
Note that at least `make regen-ast` will also need to be changed in order to reduce build times on Travis-CI.
msg302652 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-09-20 18:53
Looks like `pgen` will also have to be changed, which is more annoying as it's written in C...

Perhaps another approach can be used that will use a separate utility to do the copying if the contents changed?
msg302653 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-09-20 19:33
Posted https://github.com/python/cpython/pull/3678 which uses a separate utility.  It seems to save 1 minute off the build step on Travis-CI.
msg302654 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-09-20 19:34
Note that GNU install (`install -C`) would probably do the trick, but is not necessarily installed on the build system.
msg302662 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-20 21:57
New changeset b091bec824137f286b22084be5f8d397d21b9abb by Victor Stinner (Antoine Pitrou) in branch 'master':
bpo-31536: Avoid wholesale rebuild after `make regen-all` (#3678)
https://github.com/python/cpython/commit/b091bec824137f286b22084be5f8d397d21b9abb
msg302663 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-20 22:25
If it works well, maybe it's worth it to backport the change to the 3.6 branch. We still backport a lot of changes to 3.6. But maybe wait until 3.6.3 final is released?
msg302665 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-09-20 22:35
> If it works well, maybe it's worth it to backport the change to the 3.6 branch. We still backport a lot of changes to 3.6. But maybe wait until 3.6.3 final is released?

Fair enough. Does someone want to do the backport? :-)
msg302670 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-21 07:24
I may do the backport, or ask the bot to do it. Right now I prefer to check if everything is fine on all CI.
msg303203 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-09-28 09:34
Victor, are the buildbots fine now?
msg303204 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-28 09:36
> Victor, are the buildbots fine now?

I didn't see anything wrong on buildbots, nor Travis CI/AppVeyor on PRs. So we can now backport the "optimization". Do you want to do it, Antoine?
msg303222 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-09-28 11:21
New changeset ec3d34c5b2feb10cb4d7606a10d81c178c3afce3 by Antoine Pitrou (Miss Islington (bot)) in branch '3.6':
[3.6] bpo-31536: Avoid wholesale rebuild after `make regen-all` (GH-3678) (#3797)
https://github.com/python/cpython/commit/ec3d34c5b2feb10cb4d7606a10d81c178c3afce3
History
Date User Action Args
2017-09-28 11:21:50pitrousetstatus: open -> closed
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.6
2017-09-28 11:21:34pitrousetmessages: + msg303222
2017-09-28 10:41:38python-devsetstage: resolved -> patch review
pull_requests: + pull_request3782
2017-09-28 09:36:41vstinnersetmessages: + msg303204
2017-09-28 09:34:42pitrousetmessages: + msg303203
2017-09-21 07:24:53vstinnersetnosy: + zach.ware
2017-09-21 07:24:33vstinnersetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg302670
2017-09-20 22:35:31pitrousetmessages: + msg302665
2017-09-20 22:25:12vstinnersetmessages: + msg302663
2017-09-20 22:00:42pitrousetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2017-09-20 21:57:58vstinnersetnosy: + vstinner
messages: + msg302662
2017-09-20 19:34:15pitrousetmessages: + msg302654
2017-09-20 19:33:20pitrousetmessages: + msg302653
2017-09-20 19:28:39pitrousettitle: `make regen-opcode` triggers wholesale rebuild -> `make regen-all` triggers wholesale rebuild
2017-09-20 19:28:21pitrousetpull_requests: + pull_request3667
2017-09-20 18:53:35pitrousetmessages: + msg302652
2017-09-20 18:42:45pitrousetmessages: + msg302651
2017-09-20 18:39:32pitrousetkeywords: + patch
stage: patch review
pull_requests: + pull_request3666
2017-09-20 18:37:37pitroucreate