This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author m.sucajtys
Recipients m.sucajtys
Date 2009-08-28.08:32:32
SpamBayes Score 2.23882e-12
Marked as misclassified No
Message-id <1251448372.35.0.316865122555.issue6791@psf.upfronthosting.co.za>
In-reply-to
Content
During writing some code I discovered some behaviour of httplib. When we
connect to host, which doesn’t respond with status line, but it just
sending data, httplib may consume more and more memory, becouce when we
execute
h = httplib.HTTPConnection(‘host’)
h.conect()
h.request(‘GET’, ‘/’)
r = h.getresponse()
httplib tries to read one line from host. If host doesn’t send new line
character (‘\n’), httplib reads more and more data. On my tests httplib
could consume all of 4GB of memory and the python process was killed by
oom_killer.
The resolution is to limit maximum amount of data read on getting
response. I have performed some test:
I received 3438293 from hosts located in the network. The longest valid
response line is
HTTP/1.1 500 ( The specified Secure Sockets Layer (SSL) port is not
allowed. ISA Server is not configured to allow SSL requests from this
port. Most Web browsers use port 443 for SSL requests.  )\r\n
and it has 197 characters.
In RFC2616 in section 6.1 we have:
“The first line of a Response message is the Status-Line, consisting of
the protocol version followed by a numeric status code and its
associated textual phrase, with each element separated by SP characters.
No CR or LF is allowed except in the final CRLF sequence.
       Status-Line = HTTP-Version SP Status-Code SP Reason-Phrase CRLF
(..)The Reason-Phrase is intended to give a short textual description of
the Status-Code.”
So limiting maximum status line length to 256 characters is a solution
of this problem. It doesn’t break compatibility withc RFC 2616.

My patch was written originally on python2.4, but I’ve tested it on
python2.6:
[ms@host python2.6]$ patch --dry-run -i /home/ms/httplib.patch
patching file httplib.py
Hunk #1 succeeded at 209 (offset 54 lines).
History
Date User Action Args
2009-08-28 08:32:52m.sucajtyssetrecipients: + m.sucajtys
2009-08-28 08:32:52m.sucajtyssetmessageid: <1251448372.35.0.316865122555.issue6791@psf.upfronthosting.co.za>
2009-08-28 08:32:35m.sucajtyslinkissue6791 messages
2009-08-28 08:32:33m.sucajtyscreate