Message228818
Here are the actual numbers (instead of just from memory) running on my Windows laptop, which happens to have an SSD drive, so os.walk() with scandir is particularly good here.
python 3.4: benchmark.py -c python (all implemented in Python using ctypes):
os.walk took 1.011s, scandir.walk took 0.057s -- 17.8x as fast
python 3.4: benchmark.py -c c (using _scandir.c, so the iteration implemented in C, the DirEntry object in Python):
os.walk took 1.014s, scandir.walk took 0.039s -- 25.8x as fast
python 3.5: benchmark.py -c os (using the new all-C version in posixmodule.c):
os.walk took 0.983s, scandir.walk took 0.019s -- 52.3x as fast
So as you can see, there's still another 2x speedup to be gained from having everything in C. I know it's a bit more to review, but my thinking is if we're going to speed this baby up, let's speed it right up!
I haven't done these tests on Linux yet. |
|
Date |
User |
Action |
Args |
2014-10-08 21:55:08 | benhoyt | set | recipients:
+ benhoyt, pitrou, vstinner, giampaolo.rodola, tim.golden, abacabadabacaba, akira, socketpair |
2014-10-08 21:55:08 | benhoyt | set | messageid: <1412805308.87.0.687118975612.issue22524@psf.upfronthosting.co.za> |
2014-10-08 21:55:08 | benhoyt | link | issue22524 messages |
2014-10-08 21:55:08 | benhoyt | create | |
|