The functions urlparse.url{,un}split() and urllib{,2}.open() do not work 
together for relative, local files, due a bug in urlunsplit.

Given a file f='./rel/path/to/file.html' it can be open directly by 
urllib.open(f), but not in urllib2! as the later needs a scheme.
We can create a sound url with spilt/unspilt and a default scheme:
f2=urlparse.urlunsplit(urlparse.urlsplit(f,'file')); which works most 
cases, HOWEVER a bogus netloc is added for relative filepaths.

If have isolated this  "buggy" function, added some local testcode and 
made patch/workaround in my file 'unsplit.py' Which is included. Hope 
this will contribute to a real patch.


--Groetjes, Albert

ALbert Mietus
                                                Don't send spam mail!
Mijn missie: http://SoftwareBeterMaken.nl      product, proces & imago.
Mijn leven in het kort:        
http://albert.mietus.nl/Doc/CV_ALbert.html

There was a bug in the workaround:

    if not ( scheme == 'file' and not netloc and url[0] != '/'):
---------------------------------------------=================---

The {{{and url[0] != '/'}}} was missing (above is corrected)

The effect: split/unspilt file:///path resulted in file:/path

The bug here seems to me that urllib.urlopen() should not allow a relative file path like the one specified. f='./rel/path/to/file.html
urllib2's behavior seems proper that it is raising an Exception.

According to the RFCs the local files are to be acceessed by:
file://localhost/path/to/file
file:///path/to/file

Both are absolute paths to the file where in the second one localhost is omitted.

Let me see if urllib's urlopen be made a little stricter.

New changeset f6008e936fbc by Senthil Kumaran in branch '2.7':
Fix Issue6631 - Disallow relative files paths in urllib*.open()
http://hg.python.org/cpython/rev/f6008e936fbc

New changeset 4366c0df2c73 by Senthil Kumaran in branch '2.7':
NEWS entry for Issue6631
http://hg.python.org/cpython/rev/4366c0df2c73

New changeset 514994d7a9f2 by Senthil Kumaran in branch '3.2':
Fix  Issue6631 - Disallow relative file paths in urllib urlopen
http://hg.python.org/cpython/rev/514994d7a9f2

Sorry, why was this change backported?
Does this fix a specific issue in 2.7 or 3.2?
On the contrary, it seems to me that code which (incorrectly) used urllib.urlopen() to allow both urls and local files will suddenly break.

Actually, I saw this as a bug with urllib.urlopen and urllib2 had
exhibited proper behaviour previously. Now, both behaviour will be
consistent now.

But, you are right that an *incorrect* usage of urllib.urlopen would
break in 2.7.2. 

If we need to be lenient on that incorrect usage, then this change can
be there in 3.x series, because of urllib.request.urlopen would be
interface which users will be using and it can be reverted from 2.7.

Personally, I am +/- 0 on reverting this in 2.7. Initially, I saw this
as a bug, but later when I added tests for ValueError and checkedin,
I realized that it can break some incorrect usages, as you say.

I’m confused what the intention of this bug is. The normal urllib.request.urlopen() function (or equivalent) still allows file URLs with relative paths, in various Python versions I tried, ranging from 2.6 to 3.5:

>>> import urllib.request
>>> urllib.request.urlopen("file:README")
<addinfourl at 140061019639088 whose fp = <_io.BufferedReader name='README'>>

Passing a relative URL without a scheme seems to have never been supported, for a different reason:

>>> urllib.request.urlopen("README")
Traceback (most recent call last):
[. . .]
  File "/home/proj/python/cpython/Lib/urllib/request.py", line 321, in _parse
    raise ValueError("unknown url type: %r" % self.full_url)
ValueError: unknown url type: 'README'

Looking closer at the changes made here, they only seem to affect the urllib.request.URLopener class, which is listed as deprecated in 3.3 and never used by urlopen() as far as I know.

Personally, I would prefer to keep allowing relative paths in “file:” scheme URLs in urlopen(). This is useful e.g. if you don’t want to put the full working directory in a URL on the command line or whatever.

It is inconsistent that urljoin(), urlunsplit(), etc don’t support relative paths, but I would actually prefer that such support be added. (I think my patch for Issue 22852 would probably do the trick.) Currently, urljoin() reinterprets the path as absolute:

>>> urljoin("file:subdir/index.html", "link.html")
'file:///subdir/link.html'

test_relativelocalfile is still in place in the urllib tests, so its affecting urlopen to this point.

So I think the bug is fixed at least to the extent of the original report. I'm going to close this.