This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: make testall shows many glibc detected malloc corruptions
Type: Stage:
Components: Extension Modules Versions: Python 2.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: nnorwitz Nosy List: dfavor, georg.brandl, loewis, nnorwitz
Priority: normal Keywords:

Created on 2007-05-18 13:42 by dfavor, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
db.c nnorwitz, 2007-05-20 02:18 standalone c test case
test_get_both.py nnorwitz, 2007-05-20 02:19 standalone python test case
Messages (14)
msg32044 - (view) Author: David Favor (dfavor) Date: 2007-05-18 13:42
export _MALLOC_CHECK_=1
make testall

test_bsddb3
*** glibc detected *** ./python: free(): invalid pointer: 0x40b92de4 ***
*** glibc detected *** ./python: free(): invalid pointer: 0x40b92de4 ***
*** glibc detected *** ./python: free(): invalid pointer: 0x40b92de4 ***
*** glibc detected *** ./python: free(): invalid pointer: 0x40b92de4 ***
*** glibc detected *** ./python: free(): invalid pointer: 0x40b92de4 ***
*** glibc detected *** ./python: free(): invalid pointer: 0x40b92de4 ***

followed by deadlock errors.

malloc: using debugging hooks
blattest test_tempfile failed -- Traceback (most recent call last):
  File "/build/work/Python-2.5.1/Lib/test/test_tempfile.py", line 310, in test_noinherit
    self.failIf(retval > 0, "child process reports failure %d"%retval)
AssertionError: child process reports failure 1

test_timeout
test test_timeout failed -- Traceback (most recent call last):
  File "/build/work/Python-2.5.1/Lib/test/test_timeout.py", line 128, in testConnectTimeout
    %(_delta, self.fuzz, _timeout))
AssertionError: timeout (8.20631) is more than 2 seconds more than expected (0.001)
msg32045 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2007-05-18 17:43
Unfortunately, I cannot reproduce this problem on Debian unstable (x86, gcc 4.1.3, glibc 2.5) - 
the test passes just fine with _MALLOC_CHECK set.

Can you provide a few more details: glibc version, compiler version, Linux distribution, processor architecture, precise Python version being test, does the error happen on the first or a subsequent cycle of all tests?
msg32046 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-05-18 21:02
I don't even need to set an environment variable to get

~/devel/python> ./python Lib/test/regrtest.py -uall test_bsddb3
test_bsddb3
*** glibc detected *** ./python: free(): invalid pointer: 0x0827874c ***
[1]    17141 abort      ./python Lib/test/regrtest.py -uall test_bsddb3

(HEAD, linux x86, gcc 4.1.2, glibc 2.5)
msg32047 - (view) Author: David Favor (dfavor) Date: 2007-05-18 21:26
The os I'm running is Fedora 6 with latest patches.

gcc is 4.1.1

glibc is 2.5

I had to set MALLOC_CHECK_=1 just to get the suite to run as normal
MALLOC_CHECK_ defaults to die when something evil occurs.

I'll build gcc 4.2, rebuild and see what occurs.
msg32048 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2007-05-19 03:54
Thanks for the report David.  It would be great if you could test the change.  I couldn't find any other places that had a similar problem, but you never know.  You will need to pull out of SVN.  It's a one line change if you want to apply manually.

Committed revision 55450.
Committed revision 55451. (2.5)
msg32049 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2007-05-19 04:38
I screwed up the first checkin.  The CLEAR_DBT was in the wrong place.  The fixes are in:

Committed revision 55452. 
Committed revision 55454. (2.5)
msg32050 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2007-05-19 07:04
dfavor: I see. It is MALLOC_CHECK_, not _MALLOC_CHECK (as I used it), and not _MALLOC_CHECK_ (as you wrote). I now see that debug hooks are used, but I still cannot reproduce the problem...
msg32051 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2007-05-19 07:08
(I'm not convinced that Neal's patch could actually fix anything)

Can people who were able to reproduce this please report their bdb version also; I was using 4.4.
msg32052 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-05-19 10:31
I can confirm that the checkin didn't change anything for me.

I use bsddb version 4.5.20_p2 from Gentoo.
msg32053 - (view) Author: David Favor (dfavor) Date: 2007-05-19 15:04
My test was done with latest BerkeleyDB 4.5.20 plain (no patches).
msg32054 - (view) Author: David Favor (dfavor) Date: 2007-05-19 15:05
Just built with latest http://svn.python.org/projects/python/branches/release25-maint
pull and problem still remains.
msg32055 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2007-05-20 02:18
Sigh, 4.4.20 and 4.5.20 behave differently.  I tested on both, but botched it. :-(

It seemed like a good idea to have at least one of the buildbots to use 4.5.20.  I upgraded the g4 bot to 4.5.20, since that didn't have any version of bsddb on it.  I also installed 4.4.20 on the amd64 buildbot.  The x86 gentoo bot probably has 4.1, but it's down right now.  Martin's machine has 4.2.

Both versions (4.4 and 4.5) require the flag DB_DBT_MALLOC (get() returns an error 22 otherwise).  However only 4.4 actually allocates memory.  4.5 doesn't.  I checked in revs 55457 (2.5) which should handle this better.  Could someone please test and verify I really fixed the problem this time.  If it's really fixed, close this bug report.

The real change is to conditionally free data if it is a different pointer than we pass in (orig_data).  Don't free if they are the same pointer.

Tested 4.5 memory leaks with the leaks program and 4.4 with valgrind.
File Added: db.c
msg32056 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2007-05-20 02:19
File Added: test_get_both.py
msg32057 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2007-06-03 20:33
Seems to be really fixed now.
55457 (2.5)

Committed revision 55751. (2.6)
History
Date User Action Args
2022-04-11 14:56:24adminsetgithub: 44971
2007-05-18 13:42:47dfavorcreate