This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: fnmatch.py can appear in tracemalloc diffs
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.4, Python 3.5
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: pitrou, serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2014-07-10 19:36 by pitrou, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (4)
msg222692 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-07-10 19:36
When using some filters and comparing snapshots, fnmatch will appear in the statistic diffs, e.g.:

/home/antoine/34/lib/python3.4/fnmatch.py:70: size=824 B (+64 B), count=2 (+1), average=412 B
eeleak.py:37: size=2512 B (+0 B), count=3 (+0), average=837 B
/home/antoine/llvmpy/llvm/ee.py:109: size=976 B (+0 B), count=2 (+0), average=488 B
/home/antoine/llvmpy/llvm/core.py:401: size=904 B (+0 B), count=2 (+0), average=452 B
/home/antoine/34/lib/python3.4/sre_compile.py:488: size=856 B (+0 B), count=1 (+0), average=856 B


Perhaps it would be nice to manually cache the compiled re pattern, instead of going through fnmatch's lru_cache thing which seems to make reports less robust.
msg222696 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-07-10 20:23
> When using some filters and comparing snapshots, fnmatch will appear in the statistic diffs, e.g.:

During the design of the tracemalloc module (PEP 454), it was decided to not filter traces in the C module, but filter traces on a snapshot object. The reason is to keep the C module simple. To debug, you can analyze snapshots on a fast machine.

Can you try to filter traces?

snapshot = snapshot.filter_traces(Filter(False, tracemalloc.__file__))

In you case, the top frame is in the fnmatch module, so you probably need to trace more than one thread (ex: tracemalloc.start(10)) and apply the filter on all frames, not only on the top frame:

snapshot = snapshot.filter_traces(Filter(False, tracemalloc.__file__, all_frames=True))

You can also ignore fnmatch & sre_compile modules (blacklist, exclusive filters), or only keep traces of the directory /home/antoine/llvmpy/llvm/ (whitelist, an inclusive filter).

> Perhaps it would be nice to manually cache the compiled re pattern, instead of going through fnmatch's lru_cache thing which seems to make reports less robust.

Ah, interesting point. Would you be interested to work on that?

I tried to use tracemalloc to detect memory leaks in test.regrtest, but it's very difficult to get a reliable output. See the issue #19816.
msg246722 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-07-14 08:21
In 3.5+ lru_cache() is implemented in C. Is this issue still actual?
msg407829 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-12-06 16:21
Exposing internal caches in tracemalloc is not a bug but a deliberate choice. It's easy to write your own filters to ignore selected files (or even specific functions if you can locate them by line nubmers).
History
Date User Action Args
2022-04-11 14:58:05adminsetgithub: 66151
2021-12-06 16:21:43vstinnersetstatus: open -> closed
resolution: rejected
messages: + msg407829

stage: resolved
2015-07-14 08:21:58serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg246722
2014-07-10 20:23:31vstinnersetmessages: + msg222696
2014-07-10 19:36:07pitroucreate