A "good" solution would be one that runs the test in such a way that it doesn't fail only on Windows ;-)

There are presumably many ways that could be accomplished, including ugly ones. For example, if test_compileall is in the collection of tests to be run, and it's a parallel run, run it first by itself, before starting anything else in parallel.

No, I'm not suggesting we do that. Just trying to get across that there are _always_ ways to worm around the bug du jour.

I don't know enough about when and why CPython decides to replace .pyc files now to make another suggestion worth anyone else's time to evaluate. Victor already has something else in mind, though.