This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Threading memory leak in _shutdown_locks for non-daemon threads
Type: security Stage: resolved
Components: Versions: Python 3.9, Python 3.8, Python 3.7
Status: closed Resolution: duplicate
Dependencies: Superseder: fix for bpo-36402 (threading._shutdown() race condition) causes reference leak
View: 37788
Assigned To: Nosy List: krypticus, vstinner
Priority: normal Keywords:

Created on 2019-12-17 15:51 by krypticus, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (3)
msg358551 - (view) Author: Adam (krypticus) Date: 2019-12-17 15:51
When running 3.7, we noticed a memory leak in threading._shutdown_locks when non-deamon threads are started but "join()" or "is_alive()" is never called. Here's a test to illustrate the growth:

import threading
import time
import tracemalloc

def test_leaking_locks():
    snap1 = tracemalloc.take_snapshot()
    def print_things():
        print('.', end='')

    for x in range(500):
        t = threading.Thread(target=print_things)

    snap2 = tracemalloc.take_snapshot()
    filters = []
    for stat in snap2.filter_traces(filters).compare_to(snap1.filter_traces(filters), 'traceback')[:10]:
        print("New Bytes: {}\tTotal Bytes {}\tNew blocks: {}\tTotal blocks: {}: ".format(stat.size_diff, stat.size, stat.count_diff ,stat.count))
        for line in stat.traceback.format():


Output in v3.6.8:

New Bytes: 840	Total Bytes 840	New blocks: 1	Total blocks: 1: 
  File "/usr/local/lib/python3.6/", line 884
New Bytes: 608	Total Bytes 608	New blocks: 4	Total blocks: 4: 
  File "/usr/local/lib/python3.6/", line 387
    self.traces = _Traces(traces)
  File "/usr/local/lib/python3.6/", line 524
    return Snapshot(traces, traceback_limit)
  File "/gems/tests/integration/", line 856
    snap1 = tracemalloc.take_snapshot()
  File "/usr/local/lib/python3.6/site-packages/_pytest/", line 198
  File "/usr/local/lib/python3.6/site-packages/pluggy/", line 187
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.6/site-packages/pluggy/", line 87
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/usr/local/lib/python3.6/site-packages/pluggy/", line 93
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python3.6/site-packages/pluggy/", line 286
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/local/lib/python3.6/site-packages/_pytest/", line 1459
  File "/usr/local/lib/python3.6/site-packages/_pytest/", line 111
Output in v3.7.4:

New Bytes: 36000	Total Bytes 36000	New blocks: 1000	Total blocks: 1000: 
  File "/usr/local/lib/python3.7/", line 890
  File "/usr/local/lib/python3.7/", line 914
  File "/usr/local/lib/python3.7/", line 904
    self._tstate_lock = _set_sentinel()
New Bytes: 32768	Total Bytes 32768	New blocks: 1	Total blocks: 1: 
  File "/usr/local/lib/python3.7/", line 890
  File "/usr/local/lib/python3.7/", line 914
  File "/usr/local/lib/python3.7/", line 909

It looks like this commit didn't take into account the tstate_lock cleanup that happens in the C code, and it's not removing the _tstate_lock of completed threads from the _shutdown_locks once the thread finishes, unless the code manually calls "join()" or "is_alive()" on the thread:

Let me know if I can provide more clarity on this!
msg358597 - (view) Author: Adam (krypticus) Date: 2019-12-17 22:56
Looks like this issue might be a duplicate of
msg359476 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-06 22:51
Ok, I close this issue as a duplicate of bpo-37788.
Date User Action Args
2022-04-11 14:59:24adminsetgithub: 83255
2020-01-06 22:51:08vstinnersetstatus: open -> closed

superseder: fix for bpo-36402 (threading._shutdown() race condition) causes reference leak

nosy: + vstinner
messages: + msg359476
resolution: duplicate
stage: resolved
2019-12-17 22:56:31krypticussetmessages: + msg358597
2019-12-17 22:45:58krypticussettype: resource usage -> security
2019-12-17 15:51:34krypticuscreate