classification
Title: Automatically disable pymalloc when running under valgrind
Type: enhancement Stage:
Components: Interpreter Core Versions: Python 2.6
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, christian.heimes, dmalcolm, jamesh, lauromoura, rbcollins
Priority: normal Keywords: patch

Created on 2008-03-19 11:11 by jamesh, last changed 2009-12-03 02:58 by benjamin.peterson. This issue is now closed.

Files
File name Uploaded Description Edit
disable-pymalloc-on-valgrind.patch jamesh, 2008-03-19 11:11 disable-pymalloc-on-valgrind.patch
disable-pymalloc-on-valgrind-v3.patch jamesh, 2008-03-20 03:38 version 3
disable-pymalloc-on-valgrind-26.diff lauromoura, 2008-03-24 02:59 patch with configure option for 2.6
disable-pymalloc-on-valgrind-py26.patch jamesh, 2008-03-24 04:44 Python 2.6 version of previous patch.
disable-pymalloc-on-valgrind-py26.patch jamesh, 2008-03-27 01:21 Python 2.6 patch (v2)
disable-pymalloc-on-valgrind-py27.patch jamesh, 2009-02-04 02:41 Python 2.7 patch
Messages (10)
msg64052 - (view) Author: James Henstridge (jamesh) Date: 2008-03-19 11:11
When I want to use valgrind to check for leaks in a Python program (or
test suite), I generally want pymalloc disabled.  When not running
valgrind I generally want it enabled.

Attached is a patch that automatically bypasses the pymalloc code when
running under valgrind but leaves it enabled overwise.

It is controlled by a WITH_VALGRIND #define, but I haven't updated the
configure script to allow turning it on.  I also haven't done much in
the way of profiling to see what the overhead is when not running under
valgrind.
msg64139 - (view) Author: James Henstridge (jamesh) Date: 2008-03-20 02:56
A slightly cleaned up version of the previous patch.  I only needed to
include <valgrind/valgrind.h>.
msg64255 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-03-21 18:36
Please provide a patch for Python 2.6 that includes a ./configure
--with-valgrind option. We may consider such a patch for 2.6 and 3.0 but
definitely not for 2.5.
msg64390 - (view) Author: Lauro Moura (lauromoura) Date: 2008-03-24 02:59
Here's a patch with James changes to obmalloc and a --with-valgrind
option in configure.in.
msg64393 - (view) Author: James Henstridge (jamesh) Date: 2008-03-24 04:44
Here's the updated version of my patch (the obmalloc.c bits applied
without conflicts to the newer source tree).

The configure changes are a bit different to Lauro's ones, in that they
check for the existence of the <valgrind/valgrind.h> header if Valgrind
support was requested.
msg64575 - (view) Author: James Henstridge (jamesh) Date: 2008-03-27 01:21
An updated version of the patch.  The previous ones were missing the
valgrind check, resulting in the pymalloc code paths being executed
(which in turn cause unintialised read warnings from valgrind).
msg64837 - (view) Author: James Henstridge (jamesh) Date: 2008-04-02 05:56
There are probably a few other performance optimisations that would be
good to turn off when running under valgrind.

A big one is the tuple cache: if there are tuple reference counting
issues, they won't necessarily be seen by valgrind if the tuple is kept
and reused rather than being freed.
msg64840 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-04-02 09:42
James Henstridge schrieb:
> James Henstridge <james@jamesh.id.au> added the comment:
> 
> There are probably a few other performance optimisations that would be
> good to turn off when running under valgrind.
> 
> A big one is the tuple cache: if there are tuple reference counting
> issues, they won't necessarily be seen by valgrind if the tuple is kept
> and reused rather than being freed.

The tuple cache can't be disabled entirely. Some parts of Python and
some third party modules may depend on the fact that an empty tuple ()
is a singleton in Python.

The tuple cache could be reduced to empty tuple and some other caches
like the list and dict cache could be disabled entirely.

Christian
msg81124 - (view) Author: James Henstridge (jamesh) Date: 2009-02-04 02:41
Attached is an updated version of the patch against trunk (2.7).  It
simply fixes the conflicts that have occurred since the previous patch.
msg95923 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-12-03 02:58
Thank you. Applied in r76644.
History
Date User Action Args
2009-12-03 02:58:00benjamin.petersonsetstatus: open -> closed

nosy: + benjamin.peterson
messages: + msg95923

resolution: accepted
2009-12-03 02:42:16rbcollinssetnosy: + rbcollins
2009-11-10 16:18:45dmalcolmsetnosy: + dmalcolm
2009-02-04 02:41:18jameshsetfiles: + disable-pymalloc-on-valgrind-py27.patch
messages: + msg81124
2008-04-02 09:42:43christian.heimessetmessages: + msg64840
2008-04-02 05:56:32jameshsetmessages: + msg64837
2008-03-27 01:21:50jameshsetfiles: + disable-pymalloc-on-valgrind-py26.patch
messages: + msg64575
2008-03-24 04:45:02jameshsetfiles: + disable-pymalloc-on-valgrind-py26.patch
messages: + msg64393
2008-03-24 02:59:23lauromourasetfiles: + disable-pymalloc-on-valgrind-26.diff
nosy: + lauromoura
messages: + msg64390
2008-03-21 18:36:16christian.heimessetpriority: normal
nosy: + christian.heimes
messages: + msg64255
versions: + Python 2.6, - Python 2.5
2008-03-20 03:38:43jameshsetfiles: - disable-pymalloc-on-valgrind-v2.patch
2008-03-20 03:38:32jameshsetfiles: + disable-pymalloc-on-valgrind-v3.patch
2008-03-20 02:56:56jameshsetfiles: + disable-pymalloc-on-valgrind-v2.patch
messages: + msg64139
2008-03-19 11:11:08jameshcreate