New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
httplib: header parsing is unlimited #60241
Comments
The httplib module / package can read arbitrary amounts of data from its socket when it's parsing the HTTP header. This may lead to issues when a user connects to a broken HTTP server or something that isn't a HTTP at all. The issue can be broken up into two parts: parsing the HTTP status line parsing and parsing the remaining HTTP headers. Reading and parsing of the HTTP status line is already limited in Python 3.x. Python 2.7 and lower may read arbitrary amounts of bytes from the socket until it finds a newline char. The small patch below is a backport of the Python 3.x behavior to 2.7: --- a/Lib/httplib.py def _read_status(self):
# Initialize with Simple-Response defaults
- line = self.fp.readline()
+ line = self.fp.readline(_MAXLINE + 1)
+ if len(line) > _MAXLINE:
+ raise LineTooLong("header line")
if self.debuglevel > 0:
print "reply:", repr(line)
if not line: Both Python 2 and Python 3 accept an unlimited count of HTTP headers with a maximum length of 64k each. As headers are accumulated in an list it may consume lots of memory. I suggest that we limit the maximum amount of HTTP header lines to a sane value. How does 100 sound to you? |
New changeset 8a22a2804a66 by Christian Heimes in branch '2.7': |
The readline() limitation in _read_status() was added at some point in the 3.2 line. Python 3.1 has an unlimited readline(). |
100 headers sounds more than enough for everybody. |
CVE-2013-1752 Unbound readline() DoS vulnerabilities in Python stdlib |
Here's a patch that limits the headers to 100. If more than _MAXHEADERS headers are read, this raises exception TooMuchHeaders. The patch is for 2.7, I'll cook one for 3.2 too. |
...and here's the patch for 3.2 |
Not blocking 2.7.4 as discussed on mailing list. |
Patches LGTM but I suggest TooManyHeaders instead of TooMuchHeaders. I've tried the 3.2 patch against the latest default repo on Windows Vista and it applies cleanly. All tests passed so looks as if this could be committed. |
blocker for 2.6.9 |
Reworded TooMuch to TooMany and made a patch for 2.6 too (2.7 didn't apply cleanly there) |
As we discussed in other issues regarding the similar problem, I don't really want to introduce a new exception in a point release of 2.6. Is there any reason not to just raise HTTPException with the error message text? Code that has to work across multiple 2.6.X versions won't be able to import the new exception, and thus cannot rely on it anyway. If you agree, I'll make that change when I apply this patch. |
I'm fine with not introducing a new exception for 2.6 (or any other version for that matter), so go for it :) |
I'm just going to go ahead and commit this patch to 2.6 with the change I mentioned. Does anything else need to be done for 2.6? |
New changeset 582e5072ff89 by Barry Warsaw in branch '2.6':
|
Thanks! |
Ping. Please fix before "beta 1". |
Patch for py32 applies cleanly on 3.4 too, this should be good to go |
Third version of the 3.2 patch, this time with documentation of the exception TooManyHeaders |
New changeset e445d02e5306 by Georg Brandl in branch '3.3': |
Also merged to default. |
I presume Barry's disinclination to merge this to 2.6 with a new exception applies equally to 2.7, which is why this hasn't been merged to 2.7 yet? I'm happy to review an updated 2.7 patch that raises an HTTPException if that's what we need to keep this moving. |
Updated the patch for 2.7 to raise HTTPException instead of a new Exception. |
New changeset 5e310c6a8520 by Berker Peksag in branch '2.7': |
Thanks for the patches Jyrki and Daniel. |
Looking further, already fixed in 3.x |
Python 3.2 still receives security fixes. |
This was never discussed as a security issue. Why do you think it is? Users wasting their *own* time is different for wasting the time of a remote server in a DoS attack. |
A server can include a HTTP client. It's actually quite common these days, given the number of services which are exposed as REST APIs. |
New changeset deee87d61436 by Georg Brandl in branch '3.2': |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: