Author skrah
Recipients amaury.forgeotdarc, lemburg, mark.dickinson, skrah, vstinner
Date 2010-10-22.14:39:01
SpamBayes Score 7.84442e-07
Marked as misclassified No
Message-id <1287758352.38.0.41673734899.issue10156@psf.upfronthosting.co.za>
In-reply-to
Content
I've verified the leak manually. The cause is that global variables in
unicodeobject.c, e.g. free_list, are used before _PyUnicode_Init() is
called. Later on _PyUnicode_Init() sets these variables to NULL, losing
the allocated memory.

Here is an example of the earliest use of free_list during
_Py_ReadyTypes (),
well before _PyUnicode_Init():


Breakpoint 1, unicode_dealloc (unicode=0x1b044c0) at Objects/unicodeobject.c:392
392         switch (PyUnicode_CHECK_INTERNED(unicode)) {
(gdb) bt
#0  unicode_dealloc (unicode=0x1b044c0) at Objects/unicodeobject.c:392
#1  0x000000000044fc69 in PyUnicode_InternInPlace (p=0x7fff303852b8) at Objects/unicodeobject.c:9991
#2  0x000000000044fed3 in PyUnicode_InternFromString (cp=0x568861 "__len__") at Objects/unicodeobject.c:10025
#3  0x00000000004344d0 in init_slotdefs () at Objects/typeobject.c:5751
#4  0x0000000000434840 in add_operators (type=0x7be260) at Objects/typeobject.c:5905
#5  0x000000000042eec8 in PyType_Ready (type=0x7be260) at Objects/typeobject.c:3810
#6  0x000000000042edfc in PyType_Ready (type=0x7bde60) at Objects/typeobject.c:3774
#7  0x000000000041aa5f in _Py_ReadyTypes () at Objects/object.c:1514
#8  0x00000000004992ff in Py_InitializeEx (install_sigs=1) at Python/pythonrun.c:232
#9  0x000000000049957f in Py_Initialize () at Python/pythonrun.c:321
#10 0x00000000004b289f in Py_Main (argc=1, argv=0x1afa010) at Modules/main.c:590
#11 0x0000000000417dcc in main (argc=1, argv=0x7fff30385758) at ./Modules/python.c:59
(gdb) n
411         if (PyUnicode_CheckExact(unicode) &&
(gdb) 
414             if (unicode->length >= KEEPALIVE_SIZE_LIMIT) {
(gdb) 
419             if (unicode->defenc) {
(gdb) 
423             *(PyUnicodeObject **)unicode = free_list;
(gdb) n
424             free_list = unicode;
(gdb) n
425             numfree++;
(gdb) n
411         if (PyUnicode_CheckExact(unicode) &&


A possible fix could be to initialize the globals right at the start
in main.c. Note that there are still several Unicode API functions in
main.c before PyType_Ready has been called on the Unicode type.


With the patch, Valgrind does not show the leak any longer.
History
Date User Action Args
2010-10-22 14:39:12skrahsetrecipients: + skrah, lemburg, amaury.forgeotdarc, mark.dickinson, vstinner
2010-10-22 14:39:12skrahsetmessageid: <1287758352.38.0.41673734899.issue10156@psf.upfronthosting.co.za>
2010-10-22 14:39:03skrahlinkissue10156 messages
2010-10-22 14:39:01skrahcreate