Issue24333
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2015-05-30 18:40 by yan12125, last changed 2022-04-11 14:58 by admin. This issue is now closed.
Messages (3) | |||
---|---|---|---|
msg244483 - (view) | Author: (yan12125) * | Date: 2015-05-30 18:40 | |
This is originally my question at stackoverflow.com. (http://stackoverflow.com/q/30511341/3786245) I think it's a bug, so I posted it here. I'm trying to fetch HTTPS pages through a proxy with digest authentication. Here are my codes: import urllib.request class SimplePasswordManager(object): def __init__(self, username, password): self.username = username self.password = password def add_password(self, realm, uri, user, passwd): pass def find_user_password(self, realm, authuri): return self.username, self.password proxy_handler = urllib.request.ProxyHandler({ 'http': '<proxy server ip>', 'https': '<proxy server ip>', }) password_mgr = SimplePasswordManager('<username>', '<password>') proxy_auth_handler = urllib.request.ProxyDigestAuthHandler(passwd=password_mgr) opener = urllib.request.build_opener(proxy_auth_handler, proxy_handler) req = opener.open('http://httpbin.org/ip') print(req.read().decode('ascii')) req = opener.open('https://httpbin.org/ip') print(req.read().decode('ascii')) And the results: { "origin": "<proxy server ip>" } Traceback (most recent call last): File "/usr/lib/python3.4/urllib/request.py", line 1182, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/lib/python3.4/http/client.py", line 1088, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.4/http/client.py", line 1126, in _send_request self.endheaders(body) File "/usr/lib/python3.4/http/client.py", line 1084, in endheaders self._send_output(message_body) File "/usr/lib/python3.4/http/client.py", line 922, in _send_output self.send(msg) File "/usr/lib/python3.4/http/client.py", line 857, in send self.connect() File "/usr/lib/python3.4/http/client.py", line 1223, in connect super().connect() File "/usr/lib/python3.4/http/client.py", line 837, in connect self._tunnel() File "/usr/lib/python3.4/http/client.py", line 820, in _tunnel message.strip())) OSError: Tunnel connection failed: 407 Proxy Authentication Required During handling of the above exception, another exception occurred: Traceback (most recent call last): File "proxy_test.py", line 25, in <module> req = opener.open('https://httpbin.org/ip') File "/usr/lib/python3.4/urllib/request.py", line 463, in open response = self._open(req, data) File "/usr/lib/python3.4/urllib/request.py", line 481, in _open '_open', req) File "/usr/lib/python3.4/urllib/request.py", line 441, in _call_chain result = func(*args) File "/usr/lib/python3.4/urllib/request.py", line 1225, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.4/urllib/request.py", line 1184, in do_open raise URLError(err) urllib.error.URLError: <urlopen error Tunnel connection failed: 407 Proxy Authentication Required> Seems HTTP connection works while HTTPS not. I think it's a bug in urllib. For HTTPS connections, HTTPConnection.connect() calls HTTPConnection._tunnel(), and the latter function throws an OSError for 407 when sending a CONNECT request. There's no chance for OpenerDirector.open() to call HTTPErrorProcessor.http_response(). As a result, 407 errors are not handled correctly in ProxyDigestAuthHandler.http_error_407(). Finally, is there a workaround before this is fixed? |
|||
msg244487 - (view) | Author: (yan12125) * | Date: 2015-05-30 19:44 | |
For those who are working on this problem, my squid.conf may be helpful: --- squid.conf.default 2015-05-31 03:33:34.006361795 +0800 +++ squid.conf 2015-05-31 03:36:28.533034294 +0800 @@ -49,9 +49,15 @@ # Example rule allowing access from your local networks. # Adapt localnet in the ACL section to list your (internal) IP networks # from where browsing should be allowed -http_access allow localnet -http_access allow localhost +# http_access allow localnet +# http_access allow localhost +auth_param digest realm Proxy digest auth test +auth_param digest program /usr/lib/squid/digest_file_auth -c /etc/squid/squid-passwd +auth_param digest children 2 + +acl squid-passwd proxy_auth REQUIRED +http_access allow squid-passwd # And finally deny all other access to this proxy http_access deny all And the content of /etc/squid/squid-passwd: test_username:Proxy digest auth test:ab1e6b5de6ea6c8b072c5e513eea9c61 I'm testing for squid 3.5.5 on Arch Linux. I can test its correctness with curl: $ curl -v --proxy-user test_username:test_password --proxy localhost:3128 --proxy-digest https://httpbin.org/ip |
|||
msg244514 - (view) | Author: Martin Panter (martin.panter) * | Date: 2015-05-31 04:00 | |
There is already work done on this at Issue 7291. There is a patch there, but IMO it needs more work or a different approach. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:58:17 | admin | set | github: 68521 |
2015-05-31 04:00:10 | martin.panter | set | status: open -> closed superseder: urllib2 cannot handle https with proxy requiring auth nosy: + martin.panter messages: + msg244514 resolution: duplicate stage: resolved |
2015-05-30 19:44:32 | yan12125 | set | messages: + msg244487 |
2015-05-30 18:40:30 | yan12125 | create |