The "Check if generated files are up to date" GitHub check for PRs has been failing recently.  It may also impact local usage of "make regen-all".

Example: https://github.com/python/cpython/runs/5719012664

This may be related to gh-32061.

There's probably something racy with make.

See: https://github.com/python/cpython/runs/5712538599?check_suite_focus=true#step:10:1147

New changeset 4c116f716bd1c174d6530b9a7a5ed3863927a109 by Eric Snow in branch 'main':
bpo-47146: Eliminate a race between make regen-deepfreeze and make regen-global-objects. (gh-32162)
https://github.com/python/cpython/commit/4c116f716bd1c174d6530b9a7a5ed3863927a109

Looks like that fixed it, per https://github.com/python/cpython/pull/32134.

Looks like this is still an intermittent problem:

* https://github.com/python/cpython/pull/32195
  + failed: https://github.com/python/cpython/runs/5756616733
  + failed: https://github.com/python/cpython/runs/5753267869
  + failed: https://github.com/python/cpython/runs/5757169625
* https://github.com/python/cpython/pull/32114
  + failed: https://github.com/python/cpython/runs/5756616733
  + passed: https://github.com/python/cpython/runs/5757213346
* https://github.com/python/cpython/pull/32186
  + failed: ...
  + passed: https://github.com/python/cpython/runs/5757178754


Per Mark Shannon (on discord):

The "Check if generated files are up to date" is still failing consistently. It looks like the makefile is missing a dependency on the ./Programs/_freeze_module for targets that require /Programs/_freeze_module

The problem is likely a recursive make call "$(MAKE)" in combination with parallel execution. As far as I understand it, GNU make cannot track dependencies properly, which can lead to partial or invalid dependency graphs. One make process can modify a file while another uses the file. The permission error or ETXTBSY error are manifestation of one process creating an executable while another process is attempting to execve() it.

The paper https://accu.org/journals/overload/14/71/miller_2004/ explains the issues with recursion in great detail.

New changeset db4dada5108dd49ebca23e4559a53630a2df8447 by Eric Snow in branch 'main':
bpo-47146: Avoid Using make Recursively (gh-32206)
https://github.com/python/cpython/commit/db4dada5108dd49ebca23e4559a53630a2df8447

I re-ran jobs that had failed before I merged that gh-32206.  Several passed, but the following are still failing:

* https://github.com/python/cpython/pull/32188
   + https://github.com/python/cpython/runs/5773938424
* https://github.com/python/cpython/pull/32132
   + https://github.com/python/cpython/runs/5774054192
* https://github.com/python/cpython/pull/32177
   + https://github.com/python/cpython/runs/5773949869

Back to the drawing board...

Brandt pointed out this is consistently reproducible locally:

   make clean regen-all -j

I'll get this sorted out today.

New changeset e7bb7c2f047b4f97e4426c42ae209c969808069d by Eric Snow in branch 'main':
bpo-47146: Stop Depending On regen-deepfreeze For regen-global-objects (gh-32218)
https://github.com/python/cpython/commit/e7bb7c2f047b4f97e4426c42ae209c969808069d

I'll keep an eye on PRs for the next day or so.

Specifically: https://github.com/python/cpython/actions/workflows/build.yml.

Looks like gh-32218 worked.