Title: urllib.request.urlopen documentation falsely guarantees that a URLError will be raised on errors
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.6, Python 3.5
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Joshua.Chin, alex_thebear, docs@python, orsenthil, python-dev, r.david.murray
Priority: normal Keywords: patch

Created on 2014-11-04 17:04 by Joshua.Chin, last changed 2016-06-04 05:31 by python-dev. This issue is now closed.

File name Uploaded Description Edit
urlopen_doc.patch Joshua.Chin, 2014-11-04 17:04 review
urlopen_doc.patch Joshua.Chin, 2014-11-04 20:43 Documents when ValueError is raised review
urlopen_doc.patch alex_thebear, 2016-06-03 00:18 Note that only protocol errors raise UrlError review
Messages (9)
msg230640 - (view) Author: Joshua Chin (Joshua.Chin) * Date: 2014-11-04 17:04
The documentation for urlopen states that it "Raises URLError on errors." However, urlopen can raise a ValueError. In fact, test_urllib. urlopen_FileTests.test_relativelocalfile  specifically checks if urlopen raises a ValueError. I suggest removing the guarantee from the documentation.
msg230641 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-11-04 17:12
This is a general principle in Python.  A module may raise specific errors, but there are always other errors that may be raised.  The wording could be clarified, but it should not be removed.
msg230657 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-11-04 22:25
(Looking at your new patch...thanks for giving it a try even though I wasn't clear).

There are lots of other errors it can raise, too.  I was thinking more along the lines of "raises URLError on http protocol errors".  The problem is that's not completely accurate, either, but I don't remember exactly when it is that HTTPError can leak through.

Just for reference, in my code that calls urlopen and needs to keep running it if can't access the URL no matter what the (network-derived) reason, I catch ConnectionError, HTTPError, URLError, and IncompleteRead.  

We do not document all possible exceptions.  We document those that are specific to the module in question (which is URLError in this case) or that are part of the documented API (such as mentioning TimeoutError as what happens if the timeout specified by a timeout keyword is exceeded).

ValueErrors are a general class of errors that, when encountered, usually mean you need to fix your code (or, in the case of the SSL one you mention, check for SSL support at startup, assuming I understood correctly), rather than something you would catch around the urlopen call in a typical program.  There are, of course, occasions when you *do* catch ValueErrors in specific bits of code, but there is no practical way we can document all of the reasons ValueError might get raised, so we don't try.

All of that said, it would be lovely to have a reference somewhere (maybe a tutorial?) that went over all the possible exceptions one might get while using various network libraries, and why they might arise.  It is an issue that comes from the fact that the libraries are built on top of each other and the general python technique is that lower level errors are allowed to bubble up.  It would be a beast to keep up to date, though, which is probably one reason we don't have one.  But even that kind of guide wouldn't include ValueErrors.
msg267000 - (view) Author: Alexander Liu (alex_thebear) * Date: 2016-06-03 00:18
Fixed the docs to specifically note that only protocol related errors raise the UrlError exception.
msg267115 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-06-03 17:33
New changeset aed4b9981fca by R David Murray in branch '3.5':
#22797: clarify when URLErrors are raised by urlopen.

New changeset d085b4f779af by R David Murray in branch '3.5':
Merge: #22797: clarify when URLErrors are raised by urlopen.
msg267151 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-06-03 19:51
New changeset 8b6b6add8e47 by R David Murray in branch 'default':
psuedo merge: #22797: clarify when URLErrors are raised by urlopen.
msg267195 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-06-04 00:18
New changeset b4df20312b78 by R David Murray in branch 'default':
Clean up urlopen doc string.
msg267196 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-06-04 00:19
Thanks, Alexander.
msg267235 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-06-04 05:31
New changeset 2d69d0419879 by Martin Panter in branch 'default':
Issue #22797: Synchronize urlopen() doc string with RST documentation
Date User Action Args
2016-06-04 05:31:34python-devsetmessages: + msg267235
2016-06-04 00:19:22r.david.murraysetstatus: open -> closed
versions: + Python 3.5
messages: + msg267196

resolution: fixed
stage: resolved
2016-06-04 00:18:28python-devsetmessages: + msg267195
2016-06-03 19:51:00python-devsetmessages: + msg267151
2016-06-03 17:33:22python-devsetnosy: + python-dev
messages: + msg267115
2016-06-03 00:18:14alex_thebearsetfiles: + urlopen_doc.patch
versions: + Python 3.6, - Python 3.4
nosy: + alex_thebear

messages: + msg267000
2014-11-04 22:25:18r.david.murraysetmessages: + msg230657
2014-11-04 20:43:36Joshua.Chinsetfiles: + urlopen_doc.patch
2014-11-04 17:12:37r.david.murraysetnosy: + r.david.murray
messages: + msg230641
2014-11-04 17:04:57Joshua.Chincreate