This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients ezio.melotti, mark.dickinson, michael.foord, pitrou, rhettinger, serhiy.storchaka, vstinner
Date 2017-08-17.17:51:42
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1502992302.24.0.410861325097.issue31227@psf.upfronthosting.co.za>
In-reply-to
Content
> The exact behavior depends on the order of files in directories, on string hashes randomization, on address randomization, and on many other things out of our control.

For hash randomization, maybe we need to generate a PYTHONHASHSEED, as tox test runner does.

For the filesystem: right, it's not possible to get 100% reproductible tests, but IMHO it's worth it to make them more reliable.

> Couldn't reseeding the PRNG just add a false promise?

I'm trying to fix random failures on the Refleaks buildbots, not to promise anything :-) To be honest, at this point, I don't know if it would be enough since I'm unable to reproduce bugs...

> The success in making tests deterministic can also narrow down the coverage of the testing. Some branches that lead to failures can be never executed. Our target not just making tests always success, but catch and fix even pretty rare errors.

I know that it's though question, and that's why I opened this issue, to discuss it :-)

But I see more and more projects to get more reproductible softwares and tests:

* https://reproducible-builds.org/
* systemd big project to get more "stateless" computers, or said differently: to isolate better services
* containers which also want to isolate services, "stateless" containers
* etc.

Other test runners, like tox, also make efforts to get reproductible tests, like setting PYTHONHASHSEED.
History
Date User Action Args
2017-08-17 17:51:42vstinnersetrecipients: + vstinner, rhettinger, mark.dickinson, pitrou, ezio.melotti, michael.foord, serhiy.storchaka
2017-08-17 17:51:42vstinnersetmessageid: <1502992302.24.0.410861325097.issue31227@psf.upfronthosting.co.za>
2017-08-17 17:51:42vstinnerlinkissue31227 messages
2017-08-17 17:51:42vstinnercreate