New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
urllib may leak sensitive HTTP headers to a third-party web site #77842
Comments
After discussing it on security@python.org, it was decided to disclose it. Here is the original report: Hello Python Security Team, Looks like urllib may leak sensitive HTTP headers to third parties when handling redirects. Let's consider the following environment:
Let's then consider the following scenario:
Here is what happens next:
The problem is that urllib sends the Authorization and Cookie headers headers to http://headers.gypsyengineer.com as well. Let's imagine that a user is authenticated on a web site via one of HTTP authentication schemes (basic, digest, NTLM, SPNEGO/Kerberos), Here is a simple POC which shows the problem: import urllib.request
req = urllib.request.Request('http://httpleak.gypsyengineer.com/redirect.php?url=http://headers.gypsyengineer.com')
req.add_header('Authorization', 'Basic YWRtaW46dGVzdA==')
req.add_header('Cookie', 'This is only for httpleak.gypsyengineer.com');
with urllib.request.urlopen(req) as f:
print(f.read(2048).decode("utf-8")) Running this code results to loading http://headers.gypsyengineer.com which prints out Authorization and Cookie headers Hello, I am <b>headers.gypsyengineer.com</b></br></br> I could reproduce it with 3.5.2, and latest build of https://github.com/python/cpython If I am not missing something, it would be better if urllib filtered out sensitive HTTP headers while handling redirects. Please let me know if I wrote anything dumb and stupid, or if you have any questions :) Thanks! Artem |
According to https://stackoverflow.com/questions/1969709/how-to-forward-headers-on-http-redirect , there's nothing in the specs that mention (even the possibility) of any special request header processing. According to https://tools.ietf.org/html/rfc7231#section-6.4 , redirection targets are to be treated as effectively equal to the original URL. So, there aren't any grounds for the proposed filtering from web standards' POV. Neither are there from security POV: |
Hi Ivan, Yes, unfortunately specs don't say anything about this scenario.
I hope servers don't share this opinion :)
I am sorry that I couldn't convice you. Thank you for your reply! |
It's not about "convincing" me or anyone else. It's about showing how this will be a strict improvement. I showed that the HTTP RFC allows apps to rely on the fact that they are receiving all the headers. So filtering them arbitrarily violates the HTTP standard -- while the whole purpose of So, your suggestion is a disaster rather than improvement. |
If I am not missing something, section 6.4 of RFC 7231 doesn't explicitly discuss that all headers should be sent. I wish it did :) I think that an Authorization header for host A may make sense for host B if both A and B use the same database with user credentials. I am not sure that modern authentication mechanisms like OAuth rely on this fact (although I need to check the specs to make sure). Sending a Cookie header to a different domain looks like a violation of the same-origin policy to me. RFC 6265 says something about it https://tools.ietf.org/html/rfc6265#section-5.4 curl was recently updated to filter out Authorization headers in case of a redirect to another host. Chrome and Firefox don't sent either Authorization or Cookie headers while handling a redirect. It doesn't seem to be a disaster for them :) |
Hi, I agree with this suggestion. First, section 6.4. "Redirection 3xx" of RFC 7231 doesn't explicitly explain whether to send all headers (including Authorization). I have confirmed that some third-party-library, tool, Programing Language and web browser did NOT forward the Authorization header at redirect.
In other words, these are being on the safe side. Actually, HTTPBasicAuthHandler of urllib2 doesn't forward the Authorization header at redirect. If this suggestion is rejected, I think that it should be changed. |
Hi!, Like say Katsuhiko YOSHIDA (#11292 (comment)) this should be filter other sensitive header. I think that is reasonable if we think on a complete solution to this issue. Maybe this issue could be apply on 3.5+ version? |
Are you aware of the “add_unredirected_header” method? Maybe that is enough to avoid your problem. |
Thanks. But I think the “add_unredirected_header” is not enough. These sensitive headers should be removed only when redirecting to cross-site automatically for security like HTTPBasicAuthHandler of urllib2. In order to fulfill this requirement, I think the operation should be in HTTPRedirectHandler.redirect_request. |
According to RFC7235 (https://tools.ietf.org/html/rfc7235#section-4.1), WWW-Authenticate header is sent from server to client. And it has not credential data. Also, Cookie2 header is already obsoleted by RFC6295 (https://tools.ietf.org/html/rfc6265). So, I think that both "Authorization" and "Cookie" are enough. |
Martin, are you okay with doing this? It seems like this issue has been the topic of a few CVEs (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-20060, https://cve.mitre.org/cgi-bin/cvename.cgi?name=2018-18074, https://curl.haxx.se/docs/CVE-2018-1000007.html) as well. |
I recently ran into an error caused by this behavior while using urllib in a program that intentionally eschews third-party libraries (and thus switching to |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: