This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Dangling threads in skipped tests in test_socket
Type: resource usage Stage: resolved
Components: Tests Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: lukasz.langa, miss-islington, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2021-09-15 17:30 by serhiy.storchaka, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 28361 merged serhiy.storchaka, 2021-09-15 17:34
PR 28317 serhiy.storchaka, 2021-09-15 19:47
PR 28384 closed miss-islington, 2021-09-16 10:30
PR 28385 closed miss-islington, 2021-09-16 10:30
PR 28408 merged serhiy.storchaka, 2021-09-17 08:57
PR 28409 merged serhiy.storchaka, 2021-09-17 08:58
PR 28414 merged serhiy.storchaka, 2021-09-17 10:21
Messages (10)
msg401862 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-09-15 17:30
Dangling threads are reported when run test_socket tests which raise SkipTest in setUp() in refleak mode.

$ ./python -m test -R 3:3 test_socket -m testBCM
0:00:00 load avg: 2.53 Run tests sequentially
0:00:00 load avg: 2.53 [1/1] test_socket
beginning 6 repetitions
123456
.Warning -- threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 1)
Warning -- Dangling thread: <_MainThread(MainThread, started 139675429708416)>
.Warning -- threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 1)
Warning -- Dangling thread: <_MainThread(MainThread, started 139675429708416)>
.Warning -- threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 1)
Warning -- Dangling thread: <_MainThread(MainThread, started 139675429708416)>
...
test_socket failed (env changed)

== Tests result: SUCCESS ==

1 test altered the execution environment:
    test_socket

Total duration: 655 ms
Tests result: SUCCESS


It happens because tearDown() is not called if setUp() raises any exception (including SkipTest). If we want to execute some cleanup code it should be registered with addCleanup().
msg401934 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-09-16 10:30
New changeset 7dacb70485a0910eb298c24b4d051720ca56fb91 by Serhiy Storchaka in branch 'main':
bpo-45212: Fix dangling threads in skipped tests in test_socket (GH-28361)
https://github.com/python/cpython/commit/7dacb70485a0910eb298c24b4d051720ca56fb91
msg401941 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-09-16 12:52
def testWithTimeoutTriggeredSend(self):
        conn = self.accept_conn()
        conn.recv(88192)
+        time.sleep(1)

Was it a deliberate choice to add a sleep of 1 second? If yes, can you please add a comment to explain why?
msg401949 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-09-16 14:46
Yes, it is a deliberate choice. It is difficult to explain why it was passed without it -- dues to a specific order of calling tearDown() and callbacks registered with addCleanup() in different base classes.

Using an event object would fix it too, but:

* Re-using an existing event object (like self.done) can have side effects in future.
* Introducing a new event object has a risk of name conflicts, because the hierarchy of classes is complicated, and same name can be reused by accident.
* time.sleep() is already used in other similar tests for timeouts. We only need to do a sleep longer then the timeout on the client side (0.01).

Given all this, I decided that adding time.sleep(1) was the simplest, the most obvious, and maybe the most resistant to future human errors way. If we want to use an event object here, we will need to re-consider using time.sleep() in other tests.

What comment do you want to add?
msg402027 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-09-17 10:11
New changeset ce59ac93626004894c2b291ec599a36cfa9fb0be by Serhiy Storchaka in branch '3.10':
[3.10] bpo-45212: Fix dangling threads in skipped tests in test_socket (GH-28361) (GH-28409)
https://github.com/python/cpython/commit/ce59ac93626004894c2b291ec599a36cfa9fb0be
msg402028 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-09-17 10:11
New changeset 10c3cf78518f4b31e1527c2795694b1bcb092696 by Serhiy Storchaka in branch '3.9':
[3.9] bpo-45212: Fix dangling threads in skipped tests in test_socket (GH-28361) (GH-28408)
https://github.com/python/cpython/commit/10c3cf78518f4b31e1527c2795694b1bcb092696
msg402029 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-09-17 10:12
We can add a comment to the effect that "the wait here needs to be longer than the client-side (0.01s)". Since you already merged the change to `main`, it will be easiest to merge the backports as well and add the comment in a follow-up series.
msg402030 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-09-17 10:12
Haha, which you just did. Cool.
msg402032 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-09-17 10:24
I have merged the backported PRs because they are blockers for my other PR which is a blocker for my other PR which is a blocker for my other PR and several future PRs.
msg402052 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-09-17 13:20
New changeset 54a1760cde7bb01e5574734c389c0746762218fd by Serhiy Storchaka in branch 'main':
bpo-45212: Add a comment for time.sleep() in tests (GH-28414)
https://github.com/python/cpython/commit/54a1760cde7bb01e5574734c389c0746762218fd
History
Date User Action Args
2022-04-11 14:59:50adminsetgithub: 89375
2022-03-22 23:07:28iritkatriellinkissue22608 superseder
2021-09-17 13:20:52serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-09-17 13:20:19serhiy.storchakasetmessages: + msg402052
2021-09-17 10:24:57serhiy.storchakasetmessages: + msg402032
2021-09-17 10:21:43serhiy.storchakasetpull_requests: + pull_request26826
2021-09-17 10:12:28lukasz.langasetmessages: + msg402030
2021-09-17 10:12:00lukasz.langasetnosy: + lukasz.langa
messages: + msg402029
2021-09-17 10:11:54serhiy.storchakasetmessages: + msg402028
2021-09-17 10:11:32serhiy.storchakasetmessages: + msg402027
2021-09-17 08:58:19serhiy.storchakasetpull_requests: + pull_request26821
2021-09-17 08:57:29serhiy.storchakasetpull_requests: + pull_request26820
2021-09-16 14:46:47serhiy.storchakasetmessages: + msg401949
2021-09-16 12:52:06vstinnersetnosy: + vstinner
messages: + msg401941
2021-09-16 10:30:37miss-islingtonsetpull_requests: + pull_request26799
2021-09-16 10:30:14miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request26798
2021-09-16 10:30:09serhiy.storchakasetmessages: + msg401934
2021-09-15 19:47:25serhiy.storchakalinkissue45187 dependencies
2021-09-15 19:47:12serhiy.storchakasetpull_requests: + pull_request26789
2021-09-15 17:34:58serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request26775
2021-09-15 17:30:17serhiy.storchakacreate