This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Calling read() on HTTPError may cause KeyError in tempfile
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: corona10, sivel
Priority: normal Keywords:

Created on 2021-12-01 19:12 by sivel, last changed 2022-04-11 14:59 by admin.

Messages (1)
msg407482 - (view) Author: Matt Martz (sivel) * Date: 2021-12-01 19:12
HTTPError may not be fully initialized in some scenarios leading to an inconsistent interface.  This is documented in code at:

https://github.com/python/cpython/blob/55fe1ae9708d81b902b6fe8f6590e2a24b1bd4b0/Lib/urllib/error.py#L45-L50

Unfortunately the way this is implemented creates an inconsistent interface, and opaque code, without a number of inline comments explaining the behavior of HTTPError.

Additionally, the way that it currently works, will cause a KeyError to be raised from tempfile, which is rather confusing.

Instead of "partially initializing" the HTTPError object, I'd propose that when fp is None, that we provide it with something like io.BytesIO to fulfill the interface.  There may be other recommended solutions, I've not thought through this extensively yet.

I think I just prefer always calling self.__super_init but passing in something like io.BytesIO if fp is None

I'm willing to create the PR once I know which direction seems to make the most sense.

>>> from urllib.error import HTTPError
>>> from urllib.request import HTTPDigestAuthHandler, HTTPPasswordMgrWithDefaultRealm, build_opener
>>> passman = HTTPPasswordMgrWithDefaultRealm()
>>> passman.add_password(None, 'httpbin.org', 'user', 'wrong')
>>> opener = build_opener(HTTPDigestAuthHandler(passman))
>>> try:
...     opener.open('https://httpbin.org/digest-auth/auth/user/passwd')
... except HTTPError as e:
...     e.read()
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File ".../3.10.0/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File ".../3.10.0/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File ".../3.10.0/lib/python3.10/urllib/request.py", line 557, in error
    result = self._call_chain(*args)
  File ".../3.10.0/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File ".../3.10.0/lib/python3.10/urllib/request.py", line 1238, in http_error_401
    retry = self.http_error_auth_reqed('www-authenticate',
  File ".../3.10.0/lib/python3.10/urllib/request.py", line 1111, in http_error_auth_reqed
    raise HTTPError(req.full_url, 401, "digest auth failed",


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
  File ".../3.10.0/lib/python3.10/tempfile.py", line 473, in __getattr__
    file = self.__dict__['file']
KeyError: 'file'
History
Date User Action Args
2022-04-11 14:59:53adminsetgithub: 90113
2022-02-07 03:08:26corona10setnosy: + corona10
2022-01-15 17:09:27iritkatrielsettype: behavior
versions: - Python 3.6, Python 3.7, Python 3.8
2021-12-01 19:12:05sivelcreate