classification
Title: urllib2: request with digest auth through proxy fail
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.1, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: orsenthil Nosy List: BreamoreBoy, gregory.p.smith, jan.kollhof, jesstess, jjlee, orsenthil
Priority: normal Keywords:

Created on 2008-10-17 15:45 by jan.kollhof, last changed 2013-04-18 05:08 by orsenthil. This issue is now closed.

Files
File name Uploaded Description Edit
urllib2.py jan.kollhof, 2008-10-17 15:45 patched urllib2
Messages (5)
msg74918 - (view) Author: Jan-Klaas Kollhof (jan.kollhof) Date: 2008-10-17 15:45
When using a proxy to request URLs that require Digest Auth,
a header similar to the following is sent to the server:

Authorization: Digest ... uri="http://example.org/foobar/spam" ...

The server then responded with:

HTTP/1.1 400 Bad Request
Server: Microsoft-IIS/6.0


When not using a proxy, the header sent will look like:

Authorization: Digest ... uri="/foobar/spam" ...

and the server responded as expected with a 200 OK.

I don't know what is at fault here, urllib2 or IIS,
but patching the urllib2 to not use the full URL, when creating 
the Digest Auth. header, works.

When a proxy is being used by urllib2, the Request's get_selector() will
always return the full URL instead of just the selector.
The code that creates the Digest auth. header also uses the
get_selector() to set the "uri="-part in the auth header.
Making sure that the "uri="-part is not a full URL all seems to work.
See the patched urllib2.py attached.

I have only tested the fix against the server I had problems with,
and it seems to be working.
msg109733 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-07-09 13:30
As it's not actually known whether urllib2 or IIS is at fault, I suggest this is closed, it can always be reopened.
msg109964 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2010-07-11 05:20
This might be related to issue2202 as well.
msg186851 - (view) Author: Jessica McKellar (jesstess) * (Python triager) Date: 2013-04-13 21:13
jan.kollhof, thanks for the report and your work on this ticket!

The format of the client response to a server's digest challenge is detailed in RFC 2617, section 3.2.2: The Authorization Request Header.

The urllib2.py attachment is unfortunately the whole file and not a diff, but I think from the transcript in msg74918 that Jan is discussing the format of the digest-uri header. That field is in turn defined in RFC 2616, section 5.1.2: Request-URI.

    Request-URI    = "*" | absoluteURI | abs_path | authority

    The absoluteURI form is REQUIRED when the request is being made to a
   proxy.

So I believe urllib2 has the correct, RFC-conforming behavior here by sending the full URI through a proxy, and the issue is with IIS.

Thus, I recommend that we close this ticket as invalid.
msg187221 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2013-04-18 05:08
Yes, RFC specifically states that "The absoluteURI form is REQUIRED when the request is being made to a proxy." Thanks Jessica for making a note of that. Closing this issue as Invalid.
History
Date User Action Args
2013-04-18 05:08:29orsenthilsetstatus: open -> closed
messages: + msg187221

dependencies: - urllib2 fails against IIS 6.0 (No support for MD5-sess auth)
resolution: accepted -> not a bug
stage: test needed -> resolved
2013-04-13 21:13:46jesstesssetnosy: + jesstess
messages: + msg186851
2010-07-11 05:20:39orsenthilsetnosy: + orsenthil
messages: + msg109964

assignee: gregory.p.smith -> orsenthil
dependencies: + urllib2 fails against IIS 6.0 (No support for MD5-sess auth)
resolution: accepted
2010-07-09 13:30:32BreamoreBoysetnosy: + BreamoreBoy
messages: + msg109733
2009-02-13 01:20:54ajaksu2setnosy: + jjlee
2009-02-12 18:20:12ajaksu2setstage: test needed
2008-12-04 00:03:55gregory.p.smithsetpriority: normal
assignee: gregory.p.smith
nosy: + gregory.p.smith
versions: + Python 3.1, Python 2.7, - Python 2.5
2008-10-17 15:45:31jan.kollhofcreate