I was able to get to a website by using both IE and
FireFox but my Python code kept giving HTTP 400 Bad
request error. To debug, I set set_http_debuglevel(1) as
in the following code:

hh = urllib2.HTTPHandler() 
hh.set_http_debuglevel(1)
opener = urllib2.build_opener
(hh,urllib2.HTTPCookieProcessor(self.cj))

The printed debug messages show that this crash
happens when there is a space in the redirected
location. Here's a cut-and-paste of the relevant debug
messages (note the line starting with send that
http_error_302 is sending):

reply: 'HTTP/1.1 302 Moved Temporarily\r\n'
header: Connection: close
header: Date: Sun, 27 Feb 2005 19:52:51 GMT
header: Server: Microsoft-IIS/6.0
<---other header data-->
send: 'GET /myEmail/User?asOf=02/26/2005 11:38:12
PM&
ddn=87cb51501730
<---remaining header data-->
reply: 'HTTP/1.1 400 Bad Request\r\n'
header: Content-Type: text/html
header: Date: Sun, 27 Feb 2005 19:56:45 GMT
header: Connection: close
header: Content-Length: 20

To fix this, I first tried to encode the redirected location
in the function http_error_302() in urllib2 using the
methods urllib.quote and urllib.urlencode but to no avail
(they encode other data as well).

A temporary solution that works is to replace any space
in the redirected URL by'%20'. Below is a snippet of the
function http_error_302 in urllib2 with this suggested fix:

def http_error_302(self, req, fp, code, msg, headers):
        # Some servers (incorrectly) return multiple 
Location headers
        # (so probably same goes for URI).  Use first 
header.
        if 'location' in headers:
            newurl = headers.getheaders('location')[0]
        elif 'uri' in headers:
            newurl = headers.getheaders('uri')[0]
        else:
            return
        newurl=newurl.replace(' ','%20') # <<< TEMP FIX - 
inserting this line temporarily fixes this problem
        newurl = urlparse.urljoin(req.get_full_url(), newurl)
       <--- remainder of this function -->

Thanks!

Logged In: YES
user_id=2772

When the server sends the 302 response with 'Location:
http://example.com/url%20with%20whitespace', urllib2 seems
to work just fine.

I believe based on reading rfc2396 that a URL that contains
spaces must contain quoted spaces (%20) not literal spaces,
because space is not an "unreserved character" [2.3] and
"[d]ata must be escaped if it does not have a representation
using an unreserved character" [2.4].

Logged In: YES
user_id=261020

Sure, but if Firefox and IE do it, probably we should do the
same.

I think cookielib.escape_path(), or something similar
(perhaps without the case normalisation) is probably the
right thing to do. That's not part of any documented API; I
suppose that function or a similar one should be added to
module urlparse, and used by urllib2 and urllib when
redirecting.

As always with urllib, the fix is trivial but adding a test is hard.

Appears to be the same as bpo-918368

This bug refers to urllib2. bpo-918368 refers to urllib. It's the
same problem in each case, though.

fixed in revision 43132 ( smaller 'r' for the roundup to auto-hyperlink). :)

Sorry, I meant fixed in revision 72351.

Sorry, I meant fixed in revision 72351.

Commit in Git:

commit 690ce9b
Author: Senthil Kumaran <orsenthil@gmail.com>
Date: Tue May 5 18:41:13 2009 +0000

Fix for bpo-1153027, making Py3k changes similar to fix in bpo-918368.
This will address:
a) urllib/ in py3k,
b) urllib in py2x is addressed by bpo-918368.
c) urllib2 in py2x was already addressed in Revision 43132.