classification
Title: python-2.7.5-r3: 40 bytes in 1 blocks are definitely lost
Type: resource usage Stage:
Components: Interpreter Core Versions: Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: mmokrejs, pitrou, skrah, tim.peters, vstinner
Priority: low Keywords:

Created on 2013-08-30 13:45 by mmokrejs, last changed 2013-08-30 18:44 by tim.peters. This issue is now closed.

Files
File name Uploaded Description Edit
valgrind_python275_without-pymalloc.txt mmokrejs, 2013-08-30 17:45 valgrind output python275 without pymalloc
Messages (6)
msg196538 - (view) Author: Martin Mokrejs (mmokrejs) Date: 2013-08-30 13:45
It is not important why I had in this moment matplotlib not in sync with python itself whcih was configure using --with-pydebug ... but here I just want to show that maybe you do not test for memleaks using valgrind on import errors (maybe include such testcase into you tests).
Anyway, here is what I got:


==14007== Memcheck, a memory error detector
==14007== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==14007== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==14007== Command: /usr/bin/python2.7 blah.py
==14007== 
Traceback (most recent call last):
  File "blah.py", line 288, in <module>
    import pylab
  File "/usr/lib64/python2.7/site-packages/pylab.py", line 1, in <module>
    from matplotlib.pylab import *
  File "/usr/lib64/python2.7/site-packages/matplotlib/pylab.py", line 222, in <module>
    from matplotlib import mpl  # pulls in most modules
  File "/usr/lib64/python2.7/site-packages/matplotlib/mpl.py", line 1, in <module>
    from matplotlib import artist
  File "/usr/lib64/python2.7/site-packages/matplotlib/artist.py", line 7, in <module>
    from transforms import Bbox, IdentityTransform, TransformedBbox, \
  File "/usr/lib64/python2.7/site-packages/matplotlib/transforms.py", line 35, in <module>
    from matplotlib._path import (affine_transform, count_bboxes_overlapping_bbox,
ImportError: /usr/lib64/python2.7/site-packages/matplotlib/_path.so: undefined symbol: _PyMem_DebugFree
[100070 refs]
==14007== 
==14007== HEAP SUMMARY:
==14007==     in use at exit: 6,303,492 bytes in 31,921 blocks
==14007==   total heap usage: 1,266,299 allocs, 1,234,378 frees, 179,304,947 bytes allocated
==14007== 
==14007== 40 bytes in 1 blocks are definitely lost in loss record 167 of 3,515
==14007==    at 0x4C2C63B: malloc (vg_replace_malloc.c:270)
==14007==    by 0x4EF1E8C: PyMem_Malloc (object.c:2343)
==14007==    by 0x10064848: initialize_builtin_datetime_metadata (arraytypes.c.src:3953)
==14007==    by 0x100719E2: set_typeinfo (arraytypes.c.src:4047)
==14007==    by 0x1016354E: initmultiarray (multiarraymodule.c:4057)
==14007==    by 0x4FB2661: _PyImport_LoadDynamicModule (importdl.c:53)
==14007==    by 0x4FAE3A7: load_module (import.c:1915)
==14007==    by 0x4FB07C1: import_submodule (import.c:2700)
==14007==    by 0x4FAFCC6: load_next (import.c:2515)
==14007==    by 0x4FAED8B: import_module_level (import.c:2224)
==14007==    by 0x4FAF320: PyImport_ImportModuleLevel (import.c:2288)
==14007==    by 0x4F78862: builtin___import__ (bltinmodule.c:49)
msg196558 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-08-30 16:03
According to traces, the leak does not come from Python but matplotlib. Please report the issue to matplotlib bug tracker.
msg196560 - (view) Author: Martin Mokrejs (mmokrejs) Date: 2013-08-30 16:21
Why do you think so? My point is that this happens when import fails. But python is at fault and should handle import errors.
msg196561 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2013-08-30 16:32
The matplotlib people won't care about this one either.  matplotlib allocated the memory, and the error message at the end says it's _trying_ to call _PyMem_DebugFree (which may well be trying to release the memory), but the binaries aren't compatible:  _PyMem_DebugFree doesn't exist.  That's on you, not on matplotlib or on Python - you're mixing together binaries that can't possibly work together.

So if this bug is re-opened, it should be assigned to you to fix it ;-)
msg196564 - (view) Author: Martin Mokrejs (mmokrejs) Date: 2013-08-30 17:41
Of course I did recompile matplotlib to use the recompiled python with debug ABI. I just don't understand why if something is not leaded why a memleak has to happen.

Anyway, now my application using the recompiled matplotlib baffled this through valgrind as well. An there was no ImportError: ;-)


==17489== 40 bytes in 1 blocks are definitely lost in loss record 718 of 5,487
==17489==    at 0x4C2C63B: malloc (vg_replace_malloc.c:270)
==17489==    by 0x4EF1E8C: PyMem_Malloc (object.c:2343)
==17489==    by 0x10064848: initialize_builtin_datetime_metadata (arraytypes.c.src:3953)
==17489==    by 0x100719E2: set_typeinfo (arraytypes.c.src:4047)
==17489==    by 0x1016354E: initmultiarray (multiarraymodule.c:4057)
==17489==    by 0x4FB2661: _PyImport_LoadDynamicModule (importdl.c:53)
==17489==    by 0x4FAE3A7: load_module (import.c:1915)
==17489==    by 0x4FB07C1: import_submodule (import.c:2700)
==17489==    by 0x4FAFCC6: load_next (import.c:2515)
==17489==    by 0x4FAED8B: import_module_level (import.c:2224)
==17489==    by 0x4FAF320: PyImport_ImportModuleLevel (import.c:2288)
==17489==    by 0x4F78862: builtin___import__ (bltinmodule.c:49)


This is probably because I also added --without-pymalloc as you wanted.
msg196569 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2013-08-30 18:44
Now you have something to show the matplotlib folks - although they're not likely to get excited about leaking 40 bytes.

There is nothing Python can do about this.  matplotlib is responsible for free'ing the memory matplotlib allocates, just as Python is responsible for free'ing the memory Python allocates.  If _you_ wrote a C extension module, and did

char *p = (char *)malloc(40);

and never did a free(p) later, it would be the same kind of thing.
History
Date User Action Args
2013-08-30 18:44:06tim.peterssetmessages: + msg196569
2013-08-30 17:45:38mmokrejssetfiles: + valgrind_python275_without-pymalloc.txt
2013-08-30 17:41:01mmokrejssetmessages: + msg196564
2013-08-30 16:32:53tim.peterssetnosy: + tim.peters
messages: + msg196561
2013-08-30 16:21:54mmokrejssetmessages: + msg196560
2013-08-30 16:03:30vstinnersetstatus: open -> closed
resolution: not a bug
messages: + msg196558
2013-08-30 13:58:34pitrousetpriority: normal -> low
2013-08-30 13:56:45ezio.melottisetnosy: + pitrou, vstinner, skrah
type: resource usage
2013-08-30 13:45:47mmokrejscreate