Message64788
In py3k, buffered binary IO can be quadratic when e.g. reading a whole file.
This is a small test on 50KB, 100KB and 200KB files:
-> py3k with buffering:
./python -m timeit -s "f = open('50KB', 'rb')" "f.seek(0); f.read()"
1000 loops, best of 3: 286 usec per loop
./python -m timeit -s "f = open('100KB', 'rb')" "f.seek(0); f.read()"
1000 loops, best of 3: 1.07 msec per loop
./python -m timeit -s "f = open('200KB', 'rb')" "f.seek(0); f.read()"
100 loops, best of 3: 4.85 msec per loop
-> py3k without buffering (just the raw FileIO layer):
./python -m timeit -s "f = open('50KB', 'rb', buffering=0)" "f.seek(0);
f.read()"
10000 loops, best of 3: 46 usec per loop
./python -m timeit -s "f = open('100KB', 'rb', buffering=0)" "f.seek(0);
f.read()"
10000 loops, best of 3: 88.7 usec per loop
./python -m timeit -s "f = open('200KB', 'rb', buffering=0)" "f.seek(0);
f.read()"
10000 loops, best of 3: 156 usec per loop
-> for comparison, Python 2.5:
python -m timeit -s "f = open('50KB', 'rb')" "f.seek(0); f.read()"
10000 loops, best of 3: 34.4 usec per loop
python -m timeit -s "f = open('100KB', 'rb')" "f.seek(0); f.read()"
10000 loops, best of 3: 62.3 usec per loop
python -m timeit -s "f = open('200KB', 'rb')" "f.seek(0); f.read()"
10000 loops, best of 3: 119 usec per loop
I'm posting this issue as a reminder, but perhaps someone is already
working on this, or the goal is to translate it to C ultimately? |
|
Date |
User |
Action |
Args |
2008-03-31 18:11:25 | pitrou | set | spambayes_score: 0.0175272 -> 0.017527219 recipients:
+ pitrou |
2008-03-31 18:11:25 | pitrou | set | spambayes_score: 0.0175272 -> 0.0175272 messageid: <1206987085.65.0.944826780657.issue2523@psf.upfronthosting.co.za> |
2008-03-31 18:11:24 | pitrou | link | issue2523 messages |
2008-03-31 18:11:24 | pitrou | create | |
|