This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients amaury.forgeotdarc, belopolsky, rhettinger, vstinner
Date 2013-10-06.08:01:30
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <CAMpsgwYTdE_ens3VyMXYWRko2qnuACV0BNdS2H=aiRPSXJW8Fg@mail.gmail.com>
In-reply-to <1381045349.72.0.892338347505.issue18874@psf.upfronthosting.co.za>
Content
2013/10/6 Raymond Hettinger <report@bugs.python.org>:
> I recommend that tracemalloc focus exclusively on real allocations and ignore freelisting.   The former are interesting because they directly affect performance.  Freelists on the other hand are close to being free (as in beer).

The version of tracemalloc available on PyPI has patches to disable
completly freelists or trace usage of freelists. It is important in
Python 2 because some lists had no limit, especially int types. The
version I wrote for Python 3.4 does not anything on freelists.

Did you read the source code of the repository on python.org?

> Measuring the freelist usage (or disabling it and then measuring) makes the proposal more complex than necessary and doesn't provide useful information (freelisting is no more interesting to track than saving a value in a static variable).

I would be nice to add metrics on freelists: number and size. Using
tracemalloc, I'm trying to trace any byte. Such metrics can be
optional (ex: a new method added to the Snapshot class).

sys._debugmalloc() does already print all free lists. It should not be
hard to write these information into structures, as I did for pymalloc
statistics.

Adding a method to read memory usage once is different than
maintaining a structure (hash tables) to know that status anytime.

> Another suggestion is to expand the API to provide a way to identify a potential performance killer:  the number of reallocs() than result in a new pointer (because the data moved).

I can add a global counter for each allocator (malloc, realloc, free)
and each domain (raw, mem, obj). But I don't want to add a counter per
trace, it would have a price on performances and memory footpring.

> If you use hash table code from another project, be sure to check its licensing.

Done: see
https://mail.python.org/pipermail/python-legal-sig/2013-September/000044.html
History
Date User Action Args
2013-10-06 08:01:31vstinnersetrecipients: + vstinner, rhettinger, amaury.forgeotdarc, belopolsky
2013-10-06 08:01:31vstinnerlinkissue18874 messages
2013-10-06 08:01:30vstinnercreate