This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author lukasz.langa
Recipients lukasz.langa, pablogsal
Date 2021-07-22.14:32:29
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1626964349.84.0.902926584101.issue44708@roundup.psfhosted.org>
In-reply-to
Content
When the CPython test suite re-runs a flaky failed test, it doesn’t actually re-run just that failed test but the entire test file. The most common case for flaky tests is when networking, threading, or multiprocessing is involved. Frustratingly those are some of the largest test files we have. Re-running them takes a lot of time.

Instead of re-running the entire file, regrtest should only re-run what actually failed.

NOTE: I added 3.10 and 3.9 to this issue even though it's not a pure bugfix. The reason is that this can visibly speed up CI for open pull requests.

(Of course, it would be best to avoid flaky tests altogether and we’re working on that, but re-running what failed is a pragmatic stopgap that is necessary in times of distributed CI that we don’t fully control.)
History
Date User Action Args
2021-07-22 14:32:29lukasz.langasetrecipients: + lukasz.langa, pablogsal
2021-07-22 14:32:29lukasz.langasetmessageid: <1626964349.84.0.902926584101.issue44708@roundup.psfhosted.org>
2021-07-22 14:32:29lukasz.langalinkissue44708 messages
2021-07-22 14:32:29lukasz.langacreate