classification
Title: urllib.urlopen creates bad requests when location header of 301 redirects contain spaces
Type: Stage:
Components: Library (Lib) Versions: Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: orsenthil Nosy List: orsenthil, pr0gg3d, till
Priority: normal Keywords:

Created on 2009-07-23 21:52 by till, last changed 2009-08-15 17:50 by orsenthil. This issue is now closed.

Messages (4)
msg90864 - (view) Author: Till Maas (till) Date: 2009-07-23 21:52
If urllib.urlopen is pointed to an url that returns a redirection with a
location header that points to a url containing spaces, that are
properly urlencoded, then it creates a bad request with the spaces not
encoded in the url. Here is an example/test case:

In [1]: import urllib

In [2]:
u=urllib.urlopen("http://sourceforge.net/project/showfiles.php?group_id=16847&package_id=13374")

In [3]: u.url
Out[3]: 'http://sourceforge.net/projects/xmlrpc-c/files/Xmlrpc-c Super
Stable/download'

In [4]: u.read()
Out[4]: '<html>\r\n<head><title>400 Bad Request</title></head>\r\n<body
bgcolor="white">\r\n<center><h1>400 Bad
Request</h1></center>\r\n<hr><center>nginx/0.7.60</center>\r\n</body>\r\n</html>\r\n'

In [5]:
u=urllib.urlopen("http://sourceforge.net/projects/xmlrpc-c/files/Xmlrpc-c%20Super%20Stable/download")

In [6]: u.read()[0:100]
Out[6]: '\n\n<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0
Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xh'

In [7]:
msg91229 - (view) Author: Francesco Del Degan (pr0gg3d) Date: 2009-08-03 16:51
urllib2 does escape spaces (and other characters too):

In [20]: 
u=urllib2.urlopen("http://sourceforge.net/project/showfiles.php?
group_id=16847&package_id=13374")

In [21]: u.url
Out[21]: 'http://sourceforge.net/projects/xmlrpc-c/files/Xmlrpc-
c%20Super%20Stable/'

In [22]: u.read()[0:100]
Out[22]: '\n\n<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 
Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xh'
msg91424 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2009-08-09 01:28
This bug is already fixed in trunk (Python2.7) and py3k branch -
Issue918368.

I see that the fix was not backported, I shall do that (once the svn is up).
msg91615 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2009-08-15 17:50
Fixed and Committed revision 74462
History
Date User Action Args
2009-08-15 17:50:35orsenthilsetstatus: open -> closed
resolution: accepted -> fixed
messages: + msg91615
2009-08-09 01:28:48orsenthilsetresolution: accepted
messages: + msg91424
versions: + Python 2.6, - Python 2.5
2009-08-03 16:51:16pr0gg3dsetnosy: + pr0gg3d
messages: + msg91229
2009-07-26 03:13:40orsenthilsetassignee: orsenthil

nosy: + orsenthil
2009-07-23 21:52:48tillcreate