Author vstinner
Recipients abacabadabacaba, akira, benhoyt, giampaolo.rodola, josh.r, pitrou, socketpair, tebeka, tim.golden, vstinner
Date 2015-02-12.18:00:53
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1423764054.49.0.125407984594.issue22524@psf.upfronthosting.co.za>
In-reply-to
Content
bench_scandir.py: dummy benchmark to compare listdir+stat vs scandir+is_dir.

os.scandir() is always slower than os.listdir() on tmpfs and ext4 partitions of a local hard driver.

I will try with NFS.


Results with scandir-5.patch on Fedora 21 (Linux).

--- Using /home/haypo (ext4, hard drive) ---

Test listdir+stat vs scandir+is_dir
Temporary directory: /home/haypo/tmpji8uviyl
Create 100000 files+symlinks...
Create 10000 directories...
# entries: 210000
Benchmark...
listdir: 2187.3 ms
scandir: 1047.2 ms
listdir: 494.4 ms
scandir: 1048.1 ms
listdir: 493.0 ms
scandir: 1042.6 ms

Result:
listdir: min=493.0 ms (2.3 us per file), max=2187.3 ms (10.4 us per file)
scandir: min=1042.6 ms (5.0 us per file), max=1048.1 ms (5.0 us per file)
scandir is between 0.5x and 2.1x faster


--- Using /tmp (tmpfs, full in memory) ---

Test listdir+stat vs scandir+is_dir
Temporary directory: /tmp/tmp6_zk3mqo
Create 100000 files+symlinks...
Create 10000 directories...
# entries: 210000
Benchmark...
listdir: 405.4 ms
scandir: 1001.3 ms
listdir: 403.3 ms
scandir: 1024.2 ms
listdir: 408.1 ms
scandir: 1013.5 ms

Remove the temporary directory...

Result:
listdir: min=403.3 ms (1.9 us per file), max=408.1 ms (1.9 us per file)
scandir: min=1001.3 ms (4.8 us per file), max=1024.2 ms (4.9 us per file)
scandir is between 0.4x and 0.4x faster
History
Date User Action Args
2015-02-12 18:00:54vstinnersetrecipients: + vstinner, tebeka, pitrou, giampaolo.rodola, tim.golden, benhoyt, abacabadabacaba, akira, socketpair, josh.r
2015-02-12 18:00:54vstinnersetmessageid: <1423764054.49.0.125407984594.issue22524@psf.upfronthosting.co.za>
2015-02-12 18:00:54vstinnerlinkissue22524 messages
2015-02-12 18:00:53vstinnercreate