Author jaraco
Recipients jaraco
Date 2020-11-04.19:42:45
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1604518965.9.0.468423053621.issue42263@roundup.psfhosted.org>
In-reply-to
Content
In issue37193, I'd worked on an implementation in which a thread reference would be removed as the thread was closing, but this led to a memory leak caught by the buildbots (https://bugs.python.org/issue37193#msg380172).

As I tracked down the issue in GH-23127, I discovered that removing the reference to the thread from within the thread triggered the reference leak detection.

I've distilled that behavior into its own test which fails on master:

```
cpython master $ git diff
diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py
index e0e5406ac2..2b4924d4d0 100644
--- a/Lib/test/test_threading.py
+++ b/Lib/test/test_threading.py
@@ -443,6 +443,15 @@ def _run(self, other_ref, yet_another):
                          msg=('%d references still around' %
                               sys.getrefcount(weak_raising_cyclic_object())))
 
+    def test_remove_thread_ref_in_thread(self):
+        def remove_ref():
+            threads[:] = []
+
+        threads = [
+            threading.Thread(target=remove_ref),
+        ]
+        threads[0].start()
+
     def test_old_threading_api(self):
         # Just a quick sanity check to make sure the old method names are
         # still present
```

Running that test with refcount checks leads to this failure:

```
cpython master $ ./python.exe Tools/scripts/run_tests.py -R 3:3 test.test_threading -m test_remove_thread_ref_in_thread
/Users/jaraco/code/public/cpython/python.exe -u -W default -bb -E -m test -r -w -j 0 -u all,-largefile,-audio,-gui -R 3:3 test.test_threading -m test_remove_thread_ref_in_thread
Using random seed 8650903
0:00:00 load avg: 1.76 Run tests in parallel using 10 child processes
0:00:01 load avg: 1.78 [1/1/1] test.test_threading failed

== Tests result: FAILURE ==

1 test failed:
    test.test_threading
0:00:01 load avg: 1.78
0:00:01 load avg: 1.78 Re-running failed tests in verbose mode
0:00:01 load avg: 1.78 Re-running test.test_threading in verbose mode
beginning 6 repetitions
123456
......
test.test_threading leaked [1, 1, 1] references, sum=3
test.test_threading leaked [3, 1, 1] memory blocks, sum=5
beginning 6 repetitions
123456
test_remove_thread_ref_in_thread (test.test_threading.ThreadTests) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.001s

OK
.test_remove_thread_ref_in_thread (test.test_threading.ThreadTests) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.001s

OK
.test_remove_thread_ref_in_thread (test.test_threading.ThreadTests) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.001s

OK
.test_remove_thread_ref_in_thread (test.test_threading.ThreadTests) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.001s

OK
.test_remove_thread_ref_in_thread (test.test_threading.ThreadTests) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.001s

OK
.test_remove_thread_ref_in_thread (test.test_threading.ThreadTests) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.001s

OK
.
test.test_threading leaked [1, 1, 1] references, sum=3
test.test_threading leaked [1, 1, 1] memory blocks, sum=3
1 test failed again:
    test.test_threading

== Tests result: FAILURE then FAILURE ==

1 test failed:
    test.test_threading

1 re-run test:
    test.test_threading

Total duration: 2.0 sec
Tests result: FAILURE then FAILURE
```

Is that behavior by design? Is it simply not possible to remove a reference to a thread from within the thread?
History
Date User Action Args
2020-11-04 19:42:45jaracosetrecipients: + jaraco
2020-11-04 19:42:45jaracosetmessageid: <1604518965.9.0.468423053621.issue42263@roundup.psfhosted.org>
2020-11-04 19:42:45jaracolinkissue42263 messages
2020-11-04 19:42:45jaracocreate