classification
Title: threading timer memory leak
Type: resource usage Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder: fix for bpo-36402 (threading._shutdown() race condition) causes reference leak
View: 37788
Assigned To: Nosy List: fengjiang, martin.panter, vstinner
Priority: normal Keywords:

Created on 2021-01-28 02:24 by fengjiang, last changed 2021-09-24 15:51 by vstinner. This issue is now closed.

Messages (4)
msg385831 - (view) Author: fengjiang (fengjiang) Date: 2021-01-28 02:24
Hi,we are transfering code from python2.7 to 3.7 and find that using threading.timer will cause memory leak. It works fine in python2.7 but not 3.7. To repreduce the problem, you can simply run the code below.

While True:
    timer = threading.Timer(5, None)
    timer.start()
    timer.cancel()

you will find the memory of progress increases rapidly
msg385833 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2021-01-28 02:56
Perhaps this is caused by Issue 37788. Python 3.7.4 introduced a leak for any thread that doesn't get its "join" method called. Timer is a subclass of Thread, so to confirm, see if calling "timer.join()" after "cancel" will make the leak go away.
msg385834 - (view) Author: fengjiang (fengjiang) Date: 2021-01-28 03:11
yes, I find similar issues,the use the patch(https://github.com/python/cpython/pull/15228/files) to fix the bug
msg402574 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-09-24 15:51
I cannot reproduce the issue. IMO it has been fixed.

Moreover, you must join timers using timer.join(): a timer remains a thread.

Code:
---
import os
import threading
os.system(f"grep ^VmRSS /proc/{os.getpid()}/status")
# warmup
for n in range(10):
    timer = threading.Timer(5, None)
    timer.start()
    timer.cancel()
    timer.join()
os.system(f"grep ^VmRSS /proc/{os.getpid()}/status")
for n in range(1000):
    timer = threading.Timer(5, None)
    timer.start()
    timer.cancel()
    timer.join()
os.system(f"grep ^VmRSS /proc/{os.getpid()}/status")
---

Output on Linux with the main branch of Python (3.11):
---
VmRSS:	   10924 kB
VmRSS:	   11104 kB
VmRSS:	   11104 kB
---
History
Date User Action Args
2021-09-24 15:51:58vstinnersetstatus: open -> closed

nosy: + vstinner
messages: + msg402574

resolution: duplicate -> fixed
stage: resolved
2021-01-28 03:11:48fengjiangsetmessages: + msg385834
2021-01-28 02:56:08martin.pantersetnosy: + martin.panter
messages: + msg385833
resolution: duplicate

type: performance -> resource usage
superseder: fix for bpo-36402 (threading._shutdown() race condition) causes reference leak
2021-01-28 02:24:03fengjiangcreate