Issue8455
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2010-04-19 11:29 by vstinner, last changed 2022-04-11 14:57 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
issue8455.diff | belopolsky, 2010-06-22 00:37 | |||
issue8455-scproxy.patch | ronaldoussoren, 2010-06-22 08:08 |
Messages (15) | |||
---|---|---|---|
msg103577 - (view) | Author: STINNER Victor (vstinner) * | Date: 2010-04-19 11:29 | |
http://www.python.org/dev/buildbot/builders/x86 Tiger 3.x/builds/25/steps/test/logs/stdio test_urllib2_localnet test test_urllib2_localnet failed -- multiple errors occurred; run in verbose mode for details Re-running test test_urllib2_localnet in verbose mode test_proxy_qop_auth_int_works_or_throws_urlerror (test.test_urllib2_localnet.ProxyAuthTests) ... ok test_proxy_qop_auth_works (test.test_urllib2_localnet.ProxyAuthTests) ... ERROR test_proxy_with_bad_password_raises_httperror (test.test_urllib2_localnet.ProxyAuthTests) ... ERROR test_proxy_with_no_password_raises_httperror (test.test_urllib2_localnet.ProxyAuthTests) ... ERROR test_200 (test.test_urllib2_localnet.TestUrlopen) ... ok test_200_with_parameters (test.test_urllib2_localnet.TestUrlopen) ... ok test_404 (test.test_urllib2_localnet.TestUrlopen) ... ok test_bad_address (test.test_urllib2_localnet.TestUrlopen) ... ok test_basic (test.test_urllib2_localnet.TestUrlopen) ... ok test_chunked (test.test_urllib2_localnet.TestUrlopen) ... ok test_geturl (test.test_urllib2_localnet.TestUrlopen) ... ok test_info (test.test_urllib2_localnet.TestUrlopen) ... ok test_redirection (test.test_urllib2_localnet.TestUrlopen) ... ok test_sending_headers (test.test_urllib2_localnet.TestUrlopen) ... ok ====================================================================== ERROR: test_proxy_qop_auth_works (test.test_urllib2_localnet.ProxyAuthTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 1072, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 938, in request self._send_request(method, url, body, headers) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 976, in _send_request self.endheaders(body) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 934, in endheaders self._send_output(message_body) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 788, in _send_output self.send(msg) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 729, in send self.connect() File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 711, in connect self.timeout, self.source_address) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/socket.py", line 321, in create_connection raise error(msg) socket.error: [Errno 61] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/test_urllib2_localnet.py", line 278, in test_proxy_qop_auth_works result = self.opener.open(self.URL) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 349, in open response = self._open(req, data) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 367, in _open '_open', req) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 327, in _call_chain result = func(*args) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 1090, in http_open return self.do_open(http.client.HTTPConnection, req) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 1075, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [Errno 61] Connection refused> ====================================================================== ERROR: test_proxy_with_bad_password_raises_httperror (test.test_urllib2_localnet.ProxyAuthTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 1072, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 938, in request self._send_request(method, url, body, headers) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 976, in _send_request self.endheaders(body) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 934, in endheaders self._send_output(message_body) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 788, in _send_output self.send(msg) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 729, in send self.connect() File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 711, in connect self.timeout, self.source_address) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/socket.py", line 321, in create_connection raise error(msg) socket.error: [Errno 61] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/test_urllib2_localnet.py", line 266, in test_proxy_with_bad_password_raises_httperror self.URL) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/unittest/case.py", line 456, in assertRaises callableObj(*args, **kwargs) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 349, in open response = self._open(req, data) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 367, in _open '_open', req) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 327, in _call_chain result = func(*args) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 1090, in http_open return self.do_open(http.client.HTTPConnection, req) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 1075, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [Errno 61] Connection refused> ====================================================================== ERROR: test_proxy_with_no_password_raises_httperror (test.test_urllib2_localnet.ProxyAuthTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 1072, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 938, in request self._send_request(method, url, body, headers) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 976, in _send_request self.endheaders(body) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 934, in endheaders self._send_output(message_body) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 788, in _send_output self.send(msg) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 729, in send self.connect() File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/http/client.py", line 711, in connect self.timeout, self.source_address) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/socket.py", line 321, in create_connection raise error(msg) socket.error: [Errno 61] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/test_urllib2_localnet.py", line 272, in test_proxy_with_no_password_raises_httperror self.URL) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/unittest/case.py", line 456, in assertRaises callableObj(*args, **kwargs) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 349, in open response = self._open(req, data) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 367, in _open '_open', req) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 327, in _call_chain result = func(*args) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 1090, in http_open return self.do_open(http.client.HTTPConnection, req) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/urllib/request.py", line 1075, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [Errno 61] Connection refused> ---------------------------------------------------------------------- Ran 14 tests in 10.076s |
|||
msg103599 - (view) | Author: Ned Deily (ned.deily) * | Date: 2010-04-19 14:20 | |
This appears to be a recently introduced failure; the same failure is seen on current trunk build on 10.5 and 10.6 as well. |
|||
msg103600 - (view) | Author: STINNER Victor (vstinner) * | Date: 2010-04-19 14:22 | |
Maybe r80198 of #7154? |
|||
msg103605 - (view) | Author: Ned Deily (ned.deily) * | Date: 2010-04-19 14:29 | |
That wouldn't explain trunk failures. The _scproxy code has been in 26 and trunk for a long time. |
|||
msg103907 - (view) | Author: David Bolen (db3l) * | Date: 2010-04-21 20:25 | |
For what it's worth, I added a few debugging statements to a local trunk checkout, and it looks like the port is getting lost somewhere along the way to setup the proxy... So the refused error is accurate and because it's trying the default port 80. Note also that I think all the Proxy tests fail, even those saying they're ok, because the refused error shows up as a URLError which is what the test expects, but for other reasons. Oh, and small item I noted while doing the test - the urllib2_localnet module doesn't clean up properly during the tests - all the local servers are left listening on their ports until all the tests are done. Adding a call to self.httpd.server_close() at the end of run() in LoopbackHttpServerThread appears to fix that. Here's a sample of one test... test_proxy_qop_auth_int_works_or_throws_urlerror (test.test_urllib2_localnet.ProxyAuthTests) ... Init done Serving HTTP on localhost port 55161 Thread running Calling handle request proxy_url: http://localhost:55161 proxy_open: http None None localhost:55161 httplib _set_hostport(localhost,None) httplib _set_hostport done host=localhost port=80 httplib.connect: Connecting to ('localhost', 80) back from handle request |
|||
msg104784 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2010-05-02 11:16 | |
I'm also seeing this on OS X 10.6. It seems to have started with r80243. |
|||
msg104785 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2010-05-02 11:20 | |
To clarify, that last message was about trunk, where this test is failing for me since r80243. Adding 2.7 to the versions. |
|||
msg108330 - (view) | Author: Alexander Belopolsky (belopolsky) * | Date: 2010-06-22 00:37 | |
Apparently, the failure on OSX is due to the fact that urllib.proxy_bypass('localhost') returns True. This makes the opener to bypass the ProxyHandler that is explicitly set up to use the correct server port: def setUp(self): .. proxy_url = "http://127.0.0.1:%d" % self.server.port handler = urllib2.ProxyHandler({"http" : proxy_url}) self.proxy_digest_handler = urllib2.ProxyDigestAuthHandler() self.opener = urllib2.build_opener(handler, self.proxy_digest_handler) instead, the opener skips to the default HTTPHandler which attempts to connect on port 80 with sad results. Interestingly, >>> urllib.proxy_bypass('127.0.0.1') False So the simplest fix is s/localhost/127.0.0.1/ as done in the attached patch (issue8455.diff). |
|||
msg108331 - (view) | Author: Alexander Belopolsky (belopolsky) * | Date: 2010-06-22 00:55 | |
In fact, urllib.proxy_bypass() returns True for any simple host name (i.e. any string without '.' in it): >>> proxy_bypass('xyz') True >>> proxy_bypass('') True >>> proxy_bypass('whatever') True I think the fix I am proposing makes sense regardless of platform because proxy_url is set up numerically: proxy_url = "http://127.0.0.1:%d" % self.server.port Maybe the above should be changed to proxy_url = "%s:%d" % (self.URL, self.server.port) once URL is changed to numerical form. |
|||
msg108336 - (view) | Author: Senthil Kumaran (orsenthil) * | Date: 2010-06-22 04:23 | |
Replacing 'http://localhost' with 'http://127.0.0.1' for this test is fine. But, urllib.proxy_bypass returning for True for any string, is not correct. In the code, I see that Proxy Settings from Mac OSX system configuration is obtained by calling _get_proxy_settings(). proxy_settings = _get_proxy_settings() The is the check to to see if proxy_settings['exclude_simple'] is set. It seems that, it is set under conditions, which either might be a wrong behavior, or we should not rely on this property. |
|||
msg108341 - (view) | Author: Ronald Oussoren (ronaldoussoren) * | Date: 2010-06-22 08:08 | |
The mac-related code in urllib is correct, it uses the SystemConfiguration framework on MacOSX to get the proxy settings. That said, the code in _scproxy is not entirly correct, it makes the wrong assumption w.r.t. the value of exclude_simple when the corresponding value is not in the settings retrieved from the SystemConfiguration framework. The tests seem to make assumptions on how the proxy-detecting code functions, although I don't understand what's going on here (that is, I haven't waded through the test code and urllib to trace the code) The patch "issue-8455-scproxy.patch" ensures that the tests pass, unless the user changed their proxy settings and selected the checkbox "Exclude simple hostnames" (System Preferernces -> Network -> First network adaptor -> Advanced... -> Proxies) Please test if the patch solves the issue for you as well. NOTE: as I mention in the second paragraph the tests seem to make unwarrented assumptions about their environment. Further proof for this: when you set "http_proxy=localhost:12345" in the environment on a Unixy system the tests will fail as well (although a different subset of the tests). |
|||
msg108347 - (view) | Author: Ronald Oussoren (ronaldoussoren) * | Date: 2010-06-22 09:33 | |
I've applied my patch in r82150, which fixes the issue for me in the trunk. I'll merge the patch into the other branches if this does indeed fix the buildbot issue. |
|||
msg108405 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2010-06-22 18:10 | |
Just to confirm that with this fix test_urllib2_localnet is now passing for me. Thanks! |
|||
msg108785 - (view) | Author: Ronald Oussoren (ronaldoussoren) * | Date: 2010-06-27 12:52 | |
Applied to 2.6 (r82280), r82281 (3.2), r82282 (3.1). |
|||
msg113593 - (view) | Author: Ned Deily (ned.deily) * | Date: 2010-08-11 08:18 | |
The problem still exists on OS X 10.3. See Issue9568 for patches. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:57:00 | admin | set | github: 52701 |
2010-08-11 08:18:07 | ned.deily | set | messages: + msg113593 |
2010-06-27 12:52:27 | ronaldoussoren | set | status: open -> closed type: behavior messages: + msg108785 resolution: fixed stage: patch review -> resolved |
2010-06-22 18:10:55 | mark.dickinson | set | messages: + msg108405 |
2010-06-22 09:33:40 | ronaldoussoren | set | messages: + msg108347 |
2010-06-22 08:08:57 | ronaldoussoren | set | files:
+ issue8455-scproxy.patch messages: + msg108341 |
2010-06-22 04:23:10 | orsenthil | set | nosy:
+ orsenthil messages: + msg108336 |
2010-06-22 01:17:26 | belopolsky | set | stage: patch review |
2010-06-22 00:55:13 | belopolsky | set | messages: + msg108331 |
2010-06-22 00:37:13 | belopolsky | set | files:
+ issue8455.diff nosy: + belopolsky messages: + msg108330 keywords: + patch |
2010-06-21 22:38:53 | belopolsky | link | issue9052 superseder |
2010-05-13 09:07:51 | ronaldoussoren | set | assignee: ronaldoussoren |
2010-05-02 11:20:47 | mark.dickinson | set | messages:
+ msg104785 versions: + Python 2.7 |
2010-05-02 11:16:43 | mark.dickinson | set | nosy:
+ mark.dickinson, ronaldoussoren messages: + msg104784 |
2010-04-21 20:25:08 | db3l | set | messages: + msg103907 |
2010-04-19 14:29:02 | ned.deily | set | messages: + msg103605 |
2010-04-19 14:22:57 | vstinner | set | versions: + Python 3.1 |
2010-04-19 14:22:12 | vstinner | set | messages: + msg103600 |
2010-04-19 14:20:26 | ned.deily | set | nosy:
+ ned.deily messages: + msg103599 |
2010-04-19 11:30:09 | vstinner | set | nosy:
+ db3l |
2010-04-19 11:29:03 | vstinner | create |