Message406478
The following script exhibits the memory leak. It only happens if "dispatch_table" is set _before_ calling super().__init__, which is pretty unexpected.
import pickle
import io
import gc
import tracemalloc
tracemalloc.start(10)
snap = tracemalloc.take_snapshot()
class MyPickler(pickle.Pickler):
def __init__(self, *args, **kwargs):
# Swapping the next 2 lines "solves" the memory leak for some reason
self.dispatch_table = dict()
super().__init__(*args, **kwargs)
l=[]
for i in range(10000):
if i % 1000 == 0:
print('='*80)
snap2 = tracemalloc.take_snapshot()
stats=snap2.compare_to(snap, 'lineno')
for s in stats[:10]:
print(s)
snap = snap2
f = io.BytesIO()
MyPickler(f)
gc.collect()
The output of the last iteration is as follow. The leak of 562 kiB is apparent:
testmem.py:12: size=562 KiB (+62.5 KiB), count=9000 (+1000), average=64 B
/usr/lib/python3.10/tracemalloc.py:125: size=2376 B (-72 B), count=33 (-1), average=72 B
/usr/lib/python3.10/tracemalloc.py:129: size=72 B (+72 B), count=1 (+1), average=72 B
/usr/lib/python3.10/tracemalloc.py:502: size=252 B (+28 B), count=9 (+1), average=28 B
/usr/lib/python3.10/tracemalloc.py:498: size=2104 B (+0 B), count=36 (+0), average=58 B
/home/dourai01/Work/lisa/lisa/testmem.py:10: size=1844 B (+0 B), count=9 (+0), average=205 B
/usr/lib/python3.10/tracemalloc.py:193: size=1680 B (+0 B), count=35 (+0), average=48 B
/usr/lib/python3.10/tracemalloc.py:547: size=1256 B (+0 B), count=3 (+0), average=419 B
/usr/lib/python3.10/tracemalloc.py:226: size=832 B (+0 B), count=2 (+0), average=416 B
/usr/lib/python3.10/tracemalloc.py:173: size=800 B (+0 B), count=2 (+0), average=400 B
If "dispatch_table" is set after calling super().__init__, there is no leak anymore:
/usr/lib/python3.10/tracemalloc.py:135: size=740 B (+740 B), count=7 (+7), average=106 B
/usr/lib/python3.10/tracemalloc.py:125: size=2088 B (-656 B), count=29 (-4), average=72 B
/usr/lib/python3.10/tracemalloc.py:136: size=320 B (+320 B), count=1 (+1), average=320 B
/usr/lib/python3.10/tracemalloc.py:132: size=0 B (-256 B), count=0 (-1)
/usr/lib/python3.10/tracemalloc.py:129: size=72 B (+72 B), count=1 (+1), average=72 B
/usr/lib/python3.10/tracemalloc.py:498: size=2008 B (+48 B), count=34 (+1), average=59 B
/usr/lib/python3.10/tracemalloc.py:193: size=1584 B (+48 B), count=33 (+1), average=48 B
/usr/lib/python3.10/tracemalloc.py:502: size=196 B (-28 B), count=7 (-1), average=28 B
/usr/lib/python3.10/tracemalloc.py:126: size=84 B (+28 B), count=3 (+1), average=28 B |
|
Date |
User |
Action |
Args |
2021-11-17 16:42:29 | douglas-raillard-arm | set | recipients:
+ douglas-raillard-arm |
2021-11-17 16:42:28 | douglas-raillard-arm | set | messageid: <1637167348.82.0.904300950392.issue45830@roundup.psfhosted.org> |
2021-11-17 16:42:28 | douglas-raillard-arm | link | issue45830 messages |
2021-11-17 16:42:28 | douglas-raillard-arm | create | |
|