This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author pitrou
Recipients pitrou
Date 2008-03-31.18:11:22
SpamBayes Score 0.017527219
Marked as misclassified No
Message-id <1206987085.65.0.944826780657.issue2523@psf.upfronthosting.co.za>
In-reply-to
Content
In py3k, buffered binary IO can be quadratic when e.g. reading a whole file.
This is a small test on 50KB, 100KB and 200KB files:

-> py3k with buffering:

./python -m timeit -s "f = open('50KB', 'rb')" "f.seek(0); f.read()"
1000 loops, best of 3: 286 usec per loop
./python -m timeit -s "f = open('100KB', 'rb')" "f.seek(0); f.read()"
1000 loops, best of 3: 1.07 msec per loop
./python -m timeit -s "f = open('200KB', 'rb')" "f.seek(0); f.read()"
100 loops, best of 3: 4.85 msec per loop

-> py3k without buffering (just the raw FileIO layer):

./python -m timeit -s "f = open('50KB', 'rb', buffering=0)" "f.seek(0);
f.read()"
10000 loops, best of 3: 46 usec per loop
./python -m timeit -s "f = open('100KB', 'rb', buffering=0)" "f.seek(0);
f.read()"
10000 loops, best of 3: 88.7 usec per loop
./python -m timeit -s "f = open('200KB', 'rb', buffering=0)" "f.seek(0);
f.read()"
10000 loops, best of 3: 156 usec per loop

-> for comparison, Python 2.5:

python -m timeit -s "f = open('50KB', 'rb')" "f.seek(0); f.read()"
10000 loops, best of 3: 34.4 usec per loop
python -m timeit -s "f = open('100KB', 'rb')" "f.seek(0); f.read()"
10000 loops, best of 3: 62.3 usec per loop
python -m timeit -s "f = open('200KB', 'rb')" "f.seek(0); f.read()"
10000 loops, best of 3: 119 usec per loop

I'm posting this issue as a reminder, but perhaps someone is already
working on this, or the goal is to translate it to C ultimately?
History
Date User Action Args
2008-03-31 18:11:25pitrousetspambayes_score: 0.0175272 -> 0.017527219
recipients: + pitrou
2008-03-31 18:11:25pitrousetspambayes_score: 0.0175272 -> 0.0175272
messageid: <1206987085.65.0.944826780657.issue2523@psf.upfronthosting.co.za>
2008-03-31 18:11:24pitroulinkissue2523 messages
2008-03-31 18:11:24pitroucreate