Message300441
Antoine Pitrou: "If refleaks depend on the random seed, perhaps it's a bug worth fixing?"
I propose to change regrtest behaviour even when -R is not used, to make regrtest more deterministic.
Currently, when you run "./python -m test -r test_xxx test_yyyy", it's hard to guess the state of the RNG in test_yyy: it depends on many bytes were consumed by test_xxx. For example, if test_xxx is run on a buildbot, but skipped when I run it locally: we get a different behaviour.
I would prefer that test_yyy behaves the same when run with "./python -m test -r --randseed=5 test_xxx test_yyyy" (with test_xxx) and with "./python -m test -r --randseed=5 test_yyyy" (without test_xxx).
With my change, "./python -m test -r --randseed=5 test_yyyy test_yyyy" (sequential) and "./python -m test -r --randseed=5 -j2 test_yyyy test_yyyy" (parallel) runs test_yyy twice with the RNG in the same state.
Proposed change is part of a more global project to reduce side effects of tests, to make tests more reproductible and more "isolated". |
|
Date |
User |
Action |
Args |
2017-08-17 15:46:27 | vstinner | set | recipients:
+ vstinner, pitrou, serhiy.storchaka |
2017-08-17 15:46:27 | vstinner | set | messageid: <1502984787.66.0.502883920393.issue31227@psf.upfronthosting.co.za> |
2017-08-17 15:46:27 | vstinner | link | issue31227 messages |
2017-08-17 15:46:27 | vstinner | create | |
|