classification
Title: httplib's _read_chunked extremely slow for lots of chunks
Type: Stage:
Components: Versions: Python 3.2, Python 3.1, Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: cjw296 Nosy List: cjw296
Priority: high Keywords: easy

Created on 2009-09-04 16:08 by cjw296, last changed 2009-09-04 17:15 by cjw296. This issue is now closed.

Files
File name Uploaded Description Edit
httplib.patch.txt cjw296, 2009-09-04 16:08
Messages (3)
msg92257 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2009-09-04 16:08
As the comment in this method suggests, accumulating the value by repeated string concatenation is slow. Appending to a list speeds this 
up dramatically.

To quantify this, downloading a 110Mb file from Apache take:

~3s using Internet Explorer
2.2s using wget
30mins using the script in http://mail.python.org/pipermail/python-
dev/2009-September/091581.html

With the attached patch applied, this time drops to 2.3s.
msg92258 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2009-09-04 16:12
Fix for trunk committed in r74655.
msg92259 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2009-09-04 17:15
Fix merged to 2.6, 3.1 and 3.2 branches.
History
Date User Action Args
2009-09-04 17:15:57cjw296setstatus: open -> closed
resolution: fixed
messages: + msg92259
2009-09-04 16:12:53cjw296setmessages: + msg92258
2009-09-04 16:08:39cjw296create