Author benhoyt
Recipients abacabadabacaba, akira, benhoyt, giampaolo.rodola, pitrou, socketpair, tim.golden, vstinner
Date 2014-10-08.21:55:08
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1412805308.87.0.687118975612.issue22524@psf.upfronthosting.co.za>
In-reply-to
Content
Here are the actual numbers (instead of just from memory) running on my Windows laptop, which happens to have an SSD drive, so os.walk() with scandir is particularly good here.

python 3.4: benchmark.py -c python (all implemented in Python using ctypes):

os.walk took 1.011s, scandir.walk took 0.057s -- 17.8x as fast

python 3.4: benchmark.py -c c (using _scandir.c, so the iteration implemented in C, the DirEntry object in Python):

os.walk took 1.014s, scandir.walk took 0.039s -- 25.8x as fast

python 3.5: benchmark.py -c os (using the new all-C version in posixmodule.c):

os.walk took 0.983s, scandir.walk took 0.019s -- 52.3x as fast

So as you can see, there's still another 2x speedup to be gained from having everything in C. I know it's a bit more to review, but my thinking is if we're going to speed this baby up, let's speed it right up!

I haven't done these tests on Linux yet.
History
Date User Action Args
2014-10-08 21:55:08benhoytsetrecipients: + benhoyt, pitrou, vstinner, giampaolo.rodola, tim.golden, abacabadabacaba, akira, socketpair
2014-10-08 21:55:08benhoytsetmessageid: <1412805308.87.0.687118975612.issue22524@psf.upfronthosting.co.za>
2014-10-08 21:55:08benhoytlinkissue22524 messages
2014-10-08 21:55:08benhoytcreate