This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: httplib's _read_chunked extremely slow for lots of chunks
Type: Stage:
Components: Versions: Python 3.1, Python 3.2, Python 2.7, Python 2.6
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: cjw296 Nosy List: cjw296
Priority: high Keywords: easy

Created on 2009-09-04 16:08 by cjw296, last changed 2022-04-11 14:56 by admin. This issue is now closed.

File name Uploaded Description Edit
httplib.patch.txt cjw296, 2009-09-04 16:08
Messages (3)
msg92257 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2009-09-04 16:08
As the comment in this method suggests, accumulating the value by repeated string concatenation is slow. Appending to a list speeds this 
up dramatically.

To quantify this, downloading a 110Mb file from Apache take:

~3s using Internet Explorer
2.2s using wget
30mins using the script in

With the attached patch applied, this time drops to 2.3s.
msg92258 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2009-09-04 16:12
Fix for trunk committed in r74655.
msg92259 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2009-09-04 17:15
Fix merged to 2.6, 3.1 and 3.2 branches.
Date User Action Args
2022-04-11 14:56:52adminsetgithub: 51087
2009-09-04 17:15:57cjw296setstatus: open -> closed
resolution: fixed
messages: + msg92259
2009-09-04 16:12:53cjw296setmessages: + msg92258
2009-09-04 16:08:39cjw296create