Message397985
When the CPython test suite re-runs a flaky failed test, it doesn’t actually re-run just that failed test but the entire test file. The most common case for flaky tests is when networking, threading, or multiprocessing is involved. Frustratingly those are some of the largest test files we have. Re-running them takes a lot of time.
Instead of re-running the entire file, regrtest should only re-run what actually failed.
NOTE: I added 3.10 and 3.9 to this issue even though it's not a pure bugfix. The reason is that this can visibly speed up CI for open pull requests.
(Of course, it would be best to avoid flaky tests altogether and we’re working on that, but re-running what failed is a pragmatic stopgap that is necessary in times of distributed CI that we don’t fully control.) |
|
Date |
User |
Action |
Args |
2021-07-22 14:32:29 | lukasz.langa | set | recipients:
+ lukasz.langa, pablogsal |
2021-07-22 14:32:29 | lukasz.langa | set | messageid: <1626964349.84.0.902926584101.issue44708@roundup.psfhosted.org> |
2021-07-22 14:32:29 | lukasz.langa | link | issue44708 messages |
2021-07-22 14:32:29 | lukasz.langa | create | |
|