This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_ssl: use context manager (with) to fix ResourceWarning
Type: Stage: resolved
Components: SSL, Tests Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: vstinner Nosy List: christian.heimes, martin.panter, vstinner
Priority: normal Keywords: patch

Created on 2016-03-23 00:47 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
test_ssl_with.patch vstinner, 2016-03-23 00:47 review
Messages (9)
msg262222 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-03-23 00:47
Attached patch modifies Lib/test/test_ssl.py to use "with socket:" instead of "try: ... finally: socket.close()" or similar patterns.

Our PPC64 AIX 3.x buildbot has a broken ssl module. A lot of tests logs ResourceWarning on unclosed sockets, because errors occur before the test closes a socket.

http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/4714/steps/test/logs/stdio

test_connect_ex_error (test.test_ssl.NetworkedTests) ... ok
test_connect_with_context (test.test_ssl.NetworkedTests) ... FAIL
/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/unittest/case.py:628: ResourceWarning: unclosed <ssl.SSLSocket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('140.211.15.154', 60379), raddr=('23.253.148.168', 443)>
  outcome.errors.clear()
msg262227 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-03-23 01:52
Wouldn’t it be better to fix the underlying problem with the test cases? As I understand it, that would also avoid the warnings :)

Also, you can often use self.addCleanup(s.close), which saves a level of indentation (and associated noise in the annotate history). I used this technique in my remaining patch for Issue 25940.
msg262228 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-03-23 01:53
Also remember that sockets don’t do context management in Python 2, so this would bring more incompatibilies when merging tests between the two branches.
msg262246 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-03-23 08:02
Martin Panter added the comment:
> Wouldn’t it be better to fix the underlying problem with the test cases?

I don't feel able to fix AIX issues, and I don't think that we can fix
all issues. You still get ResourceWarning if you interrupt the test
with CTRL+c which is not a bug.

> Also, you can often use self.addCleanup(s.close), which saves a level of indentation (and associated noise in the annotate history).

It may work on some cases, but not all of them. And I like controlling
when a socket is closed: I prefer to close it inside the test, and not
sometime later with unittest magic features.

> Also remember that sockets don’t do context management in Python 2, so this would bring more incompatibilies when merging tests between the two branches.

Since this issue is not really a blocker bug, I suggest to only modify
Python 3.6.

It's a follow-up of all work on ResourceWarning (on Python 3.6) last days:
https://docs.python.org/dev/whatsnew/3.6.html#warnings
msg262317 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-03-24 01:06
My point about Python 2 was that this might add an annoying difference between the two branches. Even if you don’t apply this to Py 2, the next person adding or updating a test might be inclined to do it.

Looking closer at your patch I admit that addCleanup() wouldn’t be appropriate except for the first change.

Maybe it would be okay to leave the “try / finally” cases as they are? Moving connect() inside is good though IMO.

Anyway, I’m only mildly against this. I don’t see much benefit, but I could tolerate the downsides. I guess it’s up to you.
msg262459 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-03-25 23:11
"try: ... finally: s.close()" are "with s: ..." do the same thing, but I prefer "with" (context-manager). IMO it's more "pythonic". I prefer to see the "cleanup block" at the beginning, rather than at the end.

> My point about Python 2 was that this might add an annoying difference between the two branches. Even if you don’t apply this to Py 2, the next person adding or updating a test might be inclined to do it.

Well, there are 3 choices:

(1) Do nothing. The code rotten slowly (technical debt)
(2) Modify all actively development branches (2.7, 3.5, default)
(3) Only modify Python 3.6

Mofiying all branches means taking the (low) risk of introducing regressions. IMHO regressions occur, all the time.

I prefer the option (3) because it allows to enhance the code without taking any risk on regression.

Yeah, the code diverge, it's a natural fact.

Then you write that my changes replacing try/finally with "with" are not worth. Well, maybe you are true. Since 5 years, I convert all the code to "with" because I really like context managers. IMHO they make the code stronger and easier to read.
msg262478 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-03-26 00:51
In many cases I prefer “with” over “try / finally”. But Python 2’s sockets do not support “with” statements, so choice (2) is not possible for 2.7.

I found one potential technical problem, and maybe some redundant close() calls in the review.
msg301489 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-06 17:25
Victor, are you still interested to fix this issue?
msg301548 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-07 00:20
Sorry, I lost track of this issue. ResourceWarning only occurs on a bug, so it don't think that the issue is really important. I just close it.
History
Date User Action Args
2022-04-11 14:58:28adminsetgithub: 70799
2017-09-07 00:20:15vstinnersetstatus: open -> closed
resolution: out of date
messages: + msg301548

stage: patch review -> resolved
2017-09-06 17:25:13christian.heimessetassignee: christian.heimes -> vstinner
messages: + msg301489
versions: + Python 3.7
2016-09-15 08:12:53christian.heimessetassignee: christian.heimes

components: + SSL
nosy: + christian.heimes
2016-03-26 00:51:24martin.pantersetmessages: + msg262478
2016-03-25 23:11:45vstinnersetmessages: + msg262459
2016-03-24 01:06:23martin.pantersetmessages: + msg262317
components: + Tests
stage: patch review
2016-03-23 08:02:05vstinnersetmessages: + msg262246
2016-03-23 01:53:55martin.pantersetmessages: + msg262228
2016-03-23 01:52:26martin.pantersetnosy: + martin.panter
messages: + msg262227
2016-03-23 00:47:46vstinnercreate