Message213481
"How much this patch speeds up testing? Especially interesting results for medium-speed buildbots (about a hour)."
Quick benchmark, I only ran the test once. "Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz" (4 cores with HT: 8 logical cores) and 12 GB of RAM. I tested test_sleep.patch + more_reliable_tests.patch with TEST_SLEEP=1e-5 and TEST_SHORT_SLEEP=1e-6,
Sequence tests
==============
Command: "time ./python -m test".
Original: 18 min 40 sec.
real 18m39.936s
user 10m43.139s
sys 1m1.410s
Patched patch: 7 min 52 sec (58% faster).
real 7m51.608s
user 5m1.324s
sys 0m13.376s
The speedup would be much lower in practice, a sleep of 10 us cannot be used for buildbots. But you may use such crazy sleeps can be used on your PC to test your patches faster.
Parallel tests
==============
Command: "time ./python -m test -j10"
Original: 3 min 21 sec
real 3m20.716s
user 18m17.689s
sys 0m45.073s
Patched patch: 3 min 12 sec *but 6 tests failed* (test_asyncore, test_ftplib, test_multiprocessing_fork, test_multiprocessing_forkserver, test_multiprocessing_spawn, test_threading).
real 3m11.723s
user 19m32.329s
sys 0m48.024s
Since the test suite failed on the patched Python, it's not possible to compare performances, but the total duration is very close to the original python. The speed up looks to be null. |
|
Date |
User |
Action |
Args |
2014-03-13 21:46:17 | vstinner | set | recipients:
+ vstinner, ncoghlan, pitrou, ned.deily, r.david.murray, neologix, zach.ware, serhiy.storchaka |
2014-03-13 21:46:17 | vstinner | set | messageid: <1394747177.85.0.0574336745776.issue20910@psf.upfronthosting.co.za> |
2014-03-13 21:46:17 | vstinner | link | issue20910 messages |
2014-03-13 21:46:17 | vstinner | create | |
|