[storage@nas0 ~]$ python2.6 -c "import urlparse; print urlparse.urlsplit('http://www.famfamfam.com](http://www.famfamfam.com/', 'http', True)"
SplitResult(scheme='http', netloc='www.famfamfam.com](http:', path='//www.famfamfam.com/', query='', fragment='')
[storage@nas0 ~]$ python2.7 -c "import urlparse; print urlparse.urlsplit('http://www.famfamfam.com](http://www.famfamfam.com/', 'http', True)"
('urlsplit() - %s, scheme=%s, allow_fragments=%s', 'http://www.famfamfam.com](http://www.famfamfam.com/', 'http', True)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/apy/APy27/lib/python2.7/urlparse.py", line 184, in urlsplit
    raise ValueError("Invalid IPv6 URL")
ValueError: Invalid IPv6 URL
[storage@nas0 ~]$

via http://bitbucket.org/tarek/distribute/issue/160

Why do you think this is a regression? It looks to me like the error message is accurate. (A ']' is valid in the netloc part only when specifying an IPv6 address).

Shouldn't urlparse accept non-IPv6 URLs as well - as it always used to - when these URLs can have a single ']'?

For eg., the following URLs seems to load just fine in my browser:

http://www.google.com/search?q=foo&b=df]d&qscrl=1

And, as is the case with the django-cms PyPI page (see referred issue link in msg), such URLs seemed to be practically used in a few places.

FWIW, it should also be noted that RFC asserts square brackets to be
valid characters in the hostname portion only and that too when it is
a IPv6 url.

In the example given, at the query portion, it should be quoted (or
percent-encoded)

I couldn't find the relevant commits, but if we didn't do it, ISTM that we should backport the fix in the next 2.6 so it behaves like in 2.7.

tarek: bpo-2987 has the details on changes made for ipv6 urlparse.
Those can't be backported as it's a feature. I would rather like to see whats breaking in distutils2.

The url which resulted in this bug in distribute: "http://www.famfamfam.com](http://www.famfamfam.com/" is clearly an invalid one. What can be done is py26, looking for invalid char like '[' or ']' outside of netloc.

Senthil: thx for the pointer.

I've fixed the problem on distribute side by catching any ValueError returned by urlparse (from 2.6 or 2.7 point of view).

That said, I don't think than catching more invalid URLs in Python 2.7 should be considered as a feature.

If it's a new feature then we should have an option to explicitly parse IpV6-like URLs and leave the default behavior like it was in 2.6. If not, then it should be considered as a bug fix (meaning that Python now discards more malformed URLs) and should be backported imo.

IOW, I want to discard invalid URLs the same way no matter what the Python version is, because this is not a rule defined by Python, rather by some RFCs at the URL level.

On 2010-05-19, at 5:00 AM, Tarek Ziadé wrote:

I've fixed the problem on distribute side by catching any ValueError returned by urlparse (from 2.6 or 2.7 point of view).

Catching ValueError will catch *every* ValueError raised, rather than only the intended one: ValueError("Invalid IPv6 URL"). Can we have a custom exception for this? Generally, I am curious as to what the convention is in regards to raising standard vs custom exceptions from the standard library.

Why would you not want to catch all value errors? I assume (perhaps a bad thing) that distribute will repeat the returned error message in a more user friendly format. If a bug in urlparse returns a spurious ValueError, that will presumably be found (and then corrected) either by the test suite or by other code in addition to distribute.

The standard library should use standard exceptions unless there is a compelling reason to create a new exception. This rule of thumb has not always been followed, of course.