classification
Title: Buildbots fail when new files are added
Type: Stage:
Components: Tests Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: barry, jaraco, zach.ware
Priority: normal Keywords:

Created on 2019-05-25 13:14 by jaraco, last changed 2019-05-25 17:53 by barry.

Messages (3)
msg343478 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2019-05-25 13:14
As [reported here](https://bugs.python.org/issue34632#msg343445), I submitted a pull request that passed all tests locally and in CI, but when accepted, build bots started to fail as a result of new files having been added to the project. It seems it's necessary in Makefile.pre.in to duplicate the git manifest to declare those directories... but only for buildbot builds. I don't fully understand why that is the case, but it would be nicer if there were protections from this footgun, especially since this issue may manifest several times in the same PR.

I can think of some ways to prevent this undesirable behavior:

- Include buildbot builds in the GitHub merge request checks.
- Add a GitHub bot that checks merge requests that create directories and alert the review of directories created if Makefile.pre.in doesn't include changes reflecting those directories.
- Remove the reliance on a separate directory listing in Makefile.pre.in.
msg343482 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2019-05-25 14:08
In GH-13565, @yan12125 provides [this reference](https://github.com/python/buildmaster-config/blob/master/master/custom/factories.py) to the buildbot code that copies the code and thus imposes the requirement to declare source directories.
msg343487 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2019-05-25 15:34
The issue here is not with buildbots, but with installation on POSIX platforms.  We do now have a couple of buildbots that install Python to a local location before running the tests, which is what flushes this out (see https://github.com/python/buildmaster-config/blob/master/master/custom/factories.py#L134-L147; they simply run `make install` before running the tests).

Adding these buildbots as pre-merge CI is not currently an option due to security implications (I don't want unreviewed code running on my home network).  I have plans to eventually allow certain builders to be run pre-merge iff the `awaiting merge` label is present on the PR, but I haven't had time to work on that yet.

It might be possible to adjust one of the Travis builds to install before running tests, but that leaves some other tests un-run, which just relocates the problem.

Removing reliance on an explicit listing of directories sounds nice, but does open up the possibility of installing more than expected if run from a dirty checkout.

What about adding a "new directories added to Makefile.pre.in" check to Tools/scripts/patchcheck.py?
History
Date User Action Args
2019-05-25 17:53:01barrysetnosy: + barry
2019-05-25 15:34:09zach.waresetnosy: + zach.ware
messages: + msg343487
2019-05-25 14:08:41jaracosetmessages: + msg343482
2019-05-25 13:14:10jaracocreate