This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Fix ResourceWarning warnings in test_urllib2_localnet
Type: resource usage Stage: resolved
Components: Tests Versions: Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: martin.panter, python-dev, vstinner
Priority: normal Keywords: patch

Created on 2016-03-23 10:59 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
test_urllib2_localnet.patch vstinner, 2016-03-23 10:59 review
Messages (6)
msg262258 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-03-23 10:59
Attached patch fixes 3 ResourceWarning warnings in Lib/test/test_urllib2_localnet.py.

See also issue #26612 (test_ssl).

Example of warning logged by "./python -X tracemalloc=25  -m test -v test_urllib2_localnet ":
---
/home/haypo/prog/python/default/Lib/test/support/__init__.py:1444: ResourceWarning: unclosed <socket.socket [closed] fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>
  gc.collect()
Object allocated at (most recent call first):
  File "/home/haypo/prog/python/default/Lib/socket.py", lineno 697
    sock = socket(af, socktype, proto)
  File "/home/haypo/prog/python/default/Lib/http/client.py", lineno 898
    (self.host,self.port), self.timeout, self.source_address)
  File "/home/haypo/prog/python/default/Lib/http/client.py", lineno 926
    self.connect()
  File "/home/haypo/prog/python/default/Lib/http/client.py", lineno 983
    self.send(msg)
  File "/home/haypo/prog/python/default/Lib/http/client.py", lineno 1151
    self._send_output(message_body)
  File "/home/haypo/prog/python/default/Lib/http/client.py", lineno 1200
    self.endheaders(body)
  File "/home/haypo/prog/python/default/Lib/http/client.py", lineno 1155
    self._send_request(method, url, body, headers)
  File "/home/haypo/prog/python/default/Lib/urllib/request.py", lineno 1303
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/home/haypo/prog/python/default/Lib/urllib/request.py", lineno 1331
    return self.do_open(http.client.HTTPConnection, req)
  File "/home/haypo/prog/python/default/Lib/urllib/request.py", lineno 503
    result = func(*args)
  File "/home/haypo/prog/python/default/Lib/urllib/request.py", lineno 543
    '_open', req)
  File "/home/haypo/prog/python/default/Lib/urllib/request.py", lineno 525
    response = self._open(req, data)
  File "/home/haypo/prog/python/default/Lib/urllib/request.py", lineno 222
    return opener.open(url, data, timeout)
  File "/home/haypo/prog/python/default/Lib/test/test_urllib2_localnet.py", lineno 595
    urllib.request.urlopen(req)
  (...)
---
msg262316 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-03-24 01:03
This patch looks okay to me. I left one review suggestion.

Focussing on test_sending_headers(), the ResourceWarning seems to be only shown since revision 46329eec5515 (Issue 26590). In simpler cases, the warning would be bypassed due to Issue 19829. But in this cases it seems there is a garbage cycle which may be involved in the new warning.
msg262340 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-03-24 13:17
New changeset 54d7e9919876 by Victor Stinner in branch 'default':
Closes #26620: Fix ResourceWarning in test_urllib2_localnet
https://hg.python.org/cpython/rev/54d7e9919876
msg262341 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-03-24 13:20
Martin wrote:
> This patch looks okay to me. I left one review suggestion.

Thanks for the review, I changed this code.

While testing one more time my patch with CTRL+c, I noticed that sometimes the servers are not stopped properly and os.environ is not restored.

I made some extra changes to try to restore os.environ and try to close the server in more cases.

It's not perfect. Since unittest.TestCase.doCleanups() doesn't handle KeyboardInterrupt, there is no warranty that cleanup functions are always executed.

> Focussing on test_sending_headers(), the ResourceWarning seems to be only shown since revision 46329eec5515 (Issue 26590). In simpler cases, the warning would be bypassed due to Issue 19829. But in this cases it seems there is a garbage cycle which may be involved in the new warning.

It's possible that Python 3.5 failed to log ResourceWarning in some cases, Python 3.6 should be better on this part.
msg262393 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-03-25 00:42
This handling of KeyboardInterrupt seems like a bug with unittest IMO. But personally I don’t spend too much thought or code handling KeyboardInterrupt in tests, for similar reasons to not doing special handling of test failures in Issue 26612.
msg279178 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-10-22 03:22
New changeset d76fccbad014 by Martin Panter in branch '3.5':
Issue #26620: Fix ResourceWarning in test_urllib2_localnet
https://hg.python.org/cpython/rev/d76fccbad014
History
Date User Action Args
2022-04-11 14:58:28adminsetgithub: 70807
2016-10-22 03:22:50python-devsetmessages: + msg279178
2016-03-25 00:42:08martin.pantersetmessages: + msg262393
2016-03-24 13:20:46vstinnersetmessages: + msg262341
2016-03-24 13:17:38python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg262340

resolution: fixed
stage: resolved
2016-03-24 01:03:04martin.pantersetmessages: + msg262316
2016-03-23 10:59:11vstinnersetfiles: + test_urllib2_localnet.patch
keywords: + patch
2016-03-23 10:59:04vstinnercreate