classification
Title: test_urllib2_localnet.ProxyAuthTests fails with no_proxy and NO_PROXY env
Type: enhancement Stage: resolved
Components: Tests Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Piotr Szczepaniak, lukasz.langa, martin.panter, orsenthil, python-dev
Priority: normal Keywords: patch

Created on 2016-10-13 20:07 by Piotr Szczepaniak, last changed 2016-10-22 04:18 by martin.panter. This issue is now closed.

Files
File name Uploaded Description Edit
cpython.diff Piotr Szczepaniak, 2016-10-13 20:07 review
Messages (4)
msg278600 - (view) Author: Piotr Szczepaniak (Piotr Szczepaniak) Date: 2016-10-13 20:07
test_urllib2_localnet.ProxyAuthTests fails with no_proxy and NO_PROXY env set:
NO_PROXY=localhost,127.0.0.0/8,::1
no_proxy=localhost,127.0.0.0/8,::1

Patch attached.

Run:
./python -m unittest test.test_urllib2_localnet.ProxyAuthTests -v
test_proxy_qop_auth_int_works_or_throws_urlerror (test.test_urllib2_localnet.ProxyAuthTests) ... ok
test_proxy_qop_auth_works (test.test_urllib2_localnet.ProxyAuthTests) ... ERROR
test_proxy_with_bad_password_raises_httperror (test.test_urllib2_localnet.ProxyAuthTests) ... ERROR
test_proxy_with_no_password_raises_httperror (test.test_urllib2_localnet.ProxyAuthTests) ... ERROR

======================================================================
ERROR: test_proxy_qop_auth_works (test.test_urllib2_localnet.ProxyAuthTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/home/d0han/cpython/cpython/Lib/socket.py", line 722, in create_connection
    raise err
  File "/home/d0han/cpython/cpython/Lib/socket.py", line 713, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/d0han/cpython/cpython/Lib/test/test_urllib2_localnet.py", line 372, in test_proxy_qop_auth_works
    result = self.opener.open(self.URL)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 544, in _open
    '_open', req)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 1346, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 111] Connection refused>

======================================================================
ERROR: test_proxy_with_bad_password_raises_httperror (test.test_urllib2_localnet.ProxyAuthTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/home/d0han/cpython/cpython/Lib/socket.py", line 722, in create_connection
    raise err
  File "/home/d0han/cpython/cpython/Lib/socket.py", line 713, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/d0han/cpython/cpython/Lib/test/test_urllib2_localnet.py", line 360, in test_proxy_with_bad_password_raises_httperror
    self.URL)
  File "/home/d0han/cpython/cpython/Lib/unittest/case.py", line 728, in assertRaises
    return context.handle('assertRaises', args, kwargs)
  File "/home/d0han/cpython/cpython/Lib/unittest/case.py", line 177, in handle
    callable_obj(*args, **kwargs)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 544, in _open
    '_open', req)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 1346, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 111] Connection refused>

======================================================================
ERROR: test_proxy_with_no_password_raises_httperror (test.test_urllib2_localnet.ProxyAuthTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/d0han/cpython/cpython/Lib/http/client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/home/d0han/cpython/cpython/Lib/socket.py", line 722, in create_connection
    raise err
  File "/home/d0han/cpython/cpython/Lib/socket.py", line 713, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/d0han/cpython/cpython/Lib/test/test_urllib2_localnet.py", line 366, in test_proxy_with_no_password_raises_httperror
    self.URL)
  File "/home/d0han/cpython/cpython/Lib/unittest/case.py", line 728, in assertRaises
    return context.handle('assertRaises', args, kwargs)
  File "/home/d0han/cpython/cpython/Lib/unittest/case.py", line 177, in handle
    callable_obj(*args, **kwargs)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 544, in _open
    '_open', req)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 1346, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/home/d0han/cpython/cpython/Lib/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 111] Connection refused>

----------------------------------------------------------------------
Ran 4 tests in 3.024s

FAILED (errors=3)
msg278677 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-10-15 00:53
The test tries using ProxyHandler directly. It looks like that handler intentionally ignores the request if it matches no_proxies (Issue 6894), so I think Piotr’s approach of adjusting the tests is correct. The patch looks good to me, though I would drop that blank line in test_proxy_qop_auth_works().

It looks like setting a temporary environment variable should disable any settings from Windows registry or OS X config, so this patch should even help in those cases.
msg279177 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-10-22 03:22
New changeset 8f4424bdeadc by Martin Panter in branch '2.7':
Issue #28435: Avoid no_proxy environment variable interfering with tests
https://hg.python.org/cpython/rev/8f4424bdeadc

New changeset dc9ad44125de by Martin Panter in branch '3.5':
Issue #28435: Avoid no_proxy environment variable interfering with tests
https://hg.python.org/cpython/rev/dc9ad44125de

New changeset 05c3fbaa8fcf by Martin Panter in branch '3.6':
Issue #28435: Merge urllib test fixes from 3.5 into 3.6
https://hg.python.org/cpython/rev/05c3fbaa8fcf

New changeset 4b6e6a77e501 by Martin Panter in branch 'default':
Issue #28435: Merge urllib test fixes from 3.6
https://hg.python.org/cpython/rev/4b6e6a77e501
msg279179 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-10-22 04:18
I altered the comment (looks like it was a copy from code forcing proxies to be bypassed). Also, I didn’t port the second no_proxy fix to 2.7; it looks like there is a different workaround there which is not affected by the environment.
History
Date User Action Args
2016-10-22 04:18:40martin.pantersetstatus: open -> closed
resolution: fixed
messages: + msg279179

stage: patch review -> resolved
2016-10-22 03:22:50python-devsetnosy: + python-dev
messages: + msg279177
2016-10-15 00:54:00martin.pantersetmessages: + msg278677
versions: + Python 2.7, Python 3.5
2016-10-14 10:13:27lukasz.langasetversions: + Python 3.6
2016-10-14 03:11:04xiang.zhangsetnosy: + martin.panter
2016-10-13 21:32:03ned.deilysetnosy: + orsenthil

stage: patch review
2016-10-13 20:08:39Piotr Szczepaniaksetnosy: + lukasz.langa
2016-10-13 20:07:06Piotr Szczepaniakcreate