This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author pitrou
Recipients alexandre.vassalotti, pitrou
Date 2008-05-08.02:14:54
SpamBayes Score 0.1277725
Marked as misclassified No
Message-id <1210212896.64.0.824348724876.issue2523@psf.upfronthosting.co.za>
In-reply-to
Content
Here is a pure Python patch removing the quadratic behaviour and trying
to make read operations generally faster.

Here are some numbers:

./python -m timeit -s "f = open('50KB', 'rb')" "f.seek(0)" "while
f.read(11): pass"

-> py3k without patch: 23.6 msec per loop
-> py3k with patch: 14.5 msec per loop
-> Python 2.5: 4.72 msec per loop

./python -m timeit -s "f = open('50KB', 'rb')" "f.seek(0); f.read()"

-> py3k without patch: 284 usec per loop
-> py3k with patch: 90.1 usec per loop
-> Python 2.5: 33.8 usec per loop

./python -m timeit -s "f = open('100KB', 'rb')" "f.seek(0); f.read()"

-> py3k without patch: 828 usec per loop
-> py3k with patch: 142 usec per loop
-> Python 2.5: 62.5 usec per loop

./python -m timeit -s "f = open('200KB', 'rb')" "f.seek(0); f.read()"

-> py3k without patch: 3.67 msec per loop
-> py3k with patch: 375 usec per loop
-> Python 2.5: 131 usec per loop

And, for the record, with a 10MB file:

./python -m timeit -s "f = open('10MB', 'rb')" "f.seek(0); f.read()"

-> py3k without patch: still running after more than one minute, gave up
-> py3k with patch: 38.6 msec per loop
-> Python 2.5: 20.4 msec per loop
History
Date User Action Args
2008-05-08 02:14:56pitrousetspambayes_score: 0.127772 -> 0.1277725
recipients: + pitrou, alexandre.vassalotti
2008-05-08 02:14:56pitrousetspambayes_score: 0.127772 -> 0.127772
messageid: <1210212896.64.0.824348724876.issue2523@psf.upfronthosting.co.za>
2008-05-08 02:14:55pitroulinkissue2523 messages
2008-05-08 02:14:55pitroucreate