classification
Title: add gc/memory management tests to pybench
Type: enhancement Stage:
Components: Demos and Tools Versions: Python 3.2
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, lemburg, pitrou
Priority: normal Keywords:

Created on 2008-10-09 20:21 by pitrou, last changed 2014-09-30 09:23 by lemburg. This issue is now closed.

Messages (6)
msg74599 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-10-09 20:21
Issue #4074 suggests to me that maybe pybench should have some gc and/or
memory management tests. Marc-Andre, what do you think?
msg74604 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2008-10-09 21:19
I'm not sure whether pybench is the right tool for this.

Note that pybench disables GC per default for exactly the reasons
causing #4074 :-)

pybench already has a --with-gc switch, so it's possible to benchmark
with or without GC and see the differences.

Regarding memory management: The exact amount of used memory is hard to
determine from within a process due to the way e.g. Linux or other
modern OSes manage memory. We'd have to use special low-level system
APIs to access the true amount of allocated memory on each platform.
msg74606 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-10-09 21:26
> Note that pybench disables GC per default for exactly the reasons
> causing #4074 :-)

I know, I was thinking to enable the GC only in the GC-specific test of
course. The idea is to have a test stressing the GC heavily, such as the
example code in #4074.

The reason I suggest doing it in pybench is that it offers existing
facilities for measuring execution times, agregating results etc.

> Regarding memory management: The exact amount of used memory is hard to
> determine from within a process due to the way e.g. Linux or other
> modern OSes manage memory.

I was not thinking about measuring the amount of used memory but rather
the time spent in specific tests which would focus on the CPU cost of
memory allocation rather than computation or control flow. Something
like allocating lots of strings or tuples of various sizes, then
releasing them in various orders.
msg74608 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2008-10-09 23:01
I'll follow up on this next week.
msg227765 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-09-28 16:24
Antoine or Marc-Andre will either of you follow up on this as the last message msg74608 was nearly six years ago?
msg227872 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2014-09-30 09:23
I think it's better to keep pybench speed performance oriented and rather use a new tool for memory performance tests.

I've recently done a lightning talk at PyCon UK on the subject and found the current tools we have for memory testing a bit underdeveloped:

http://www.egenix.com/library/presentations/PyCon-UK-2014-When-performance-matters/

As s result, I wrote some support helpers for this, which can be found in perftools.py on:

https://github.com/egenix/when-performance-matters

Perhaps this will some day turn into a pymembench :-)

For now, I'm closing the ticket.
History
Date User Action Args
2014-09-30 09:23:14lemburgsetstatus: open -> closed

messages: + msg227872
2014-09-28 16:24:11BreamoreBoysetnosy: + BreamoreBoy
messages: + msg227765
2010-08-09 03:11:11terry.reedysetversions: + Python 3.2, - Python 3.1, Python 2.7
2008-10-09 23:01:16lemburgsetmessages: + msg74608
2008-10-09 21:26:55pitrousetmessages: + msg74606
2008-10-09 21:19:57lemburgsetmessages: + msg74604
2008-10-09 20:21:10pitroucreate