export _MALLOC_CHECK_=1
make testall

test_bsddb3
*** glibc detected *** ./python: free(): invalid pointer: 0x40b92de4 ***
*** glibc detected *** ./python: free(): invalid pointer: 0x40b92de4 ***
*** glibc detected *** ./python: free(): invalid pointer: 0x40b92de4 ***
*** glibc detected *** ./python: free(): invalid pointer: 0x40b92de4 ***
*** glibc detected *** ./python: free(): invalid pointer: 0x40b92de4 ***
*** glibc detected *** ./python: free(): invalid pointer: 0x40b92de4 ***

followed by deadlock errors.

malloc: using debugging hooks
blattest test_tempfile failed -- Traceback (most recent call last):
  File "/build/work/Python-2.5.1/Lib/test/test_tempfile.py", line 310, in test_noinherit
    self.failIf(retval > 0, "child process reports failure %d"%retval)
AssertionError: child process reports failure 1

test_timeout
test test_timeout failed -- Traceback (most recent call last):
  File "/build/work/Python-2.5.1/Lib/test/test_timeout.py", line 128, in testConnectTimeout
    %(_delta, self.fuzz, _timeout))
AssertionError: timeout (8.20631) is more than 2 seconds more than expected (0.001)

Unfortunately, I cannot reproduce this problem on Debian unstable (x86, gcc 4.1.3, glibc 2.5) - 
the test passes just fine with _MALLOC_CHECK set.

Can you provide a few more details: glibc version, compiler version, Linux distribution, processor architecture, precise Python version being test, does the error happen on the first or a subsequent cycle of all tests?

I don't even need to set an environment variable to get

~/devel/python> ./python Lib/test/regrtest.py -uall test_bsddb3
test_bsddb3
*** glibc detected *** ./python: free(): invalid pointer: 0x0827874c ***
[1]    17141 abort      ./python Lib/test/regrtest.py -uall test_bsddb3

(HEAD, linux x86, gcc 4.1.2, glibc 2.5)

The os I'm running is Fedora 6 with latest patches.

gcc is 4.1.1

glibc is 2.5

I had to set MALLOC_CHECK_=1 just to get the suite to run as normal
MALLOC_CHECK_ defaults to die when something evil occurs.

I'll build gcc 4.2, rebuild and see what occurs.

Thanks for the report David.  It would be great if you could test the change.  I couldn't find any other places that had a similar problem, but you never know.  You will need to pull out of SVN.  It's a one line change if you want to apply manually.

Committed revision 55450.
Committed revision 55451. (2.5)

I screwed up the first checkin.  The CLEAR_DBT was in the wrong place.  The fixes are in:

Committed revision 55452. 
Committed revision 55454. (2.5)

dfavor: I see. It is MALLOC_CHECK_, not _MALLOC_CHECK (as I used it), and not _MALLOC_CHECK_ (as you wrote). I now see that debug hooks are used, but I still cannot reproduce the problem...

(I'm not convinced that Neal's patch could actually fix anything)

Can people who were able to reproduce this please report their bdb version also; I was using 4.4.

I can confirm that the checkin didn't change anything for me.

I use bsddb version 4.5.20_p2 from Gentoo.

My test was done with latest BerkeleyDB 4.5.20 plain (no patches).

Just built with latest http://svn.python.org/projects/python/branches/release25-maint
pull and problem still remains.

Sigh, 4.4.20 and 4.5.20 behave differently.  I tested on both, but botched it. :-(

It seemed like a good idea to have at least one of the buildbots to use 4.5.20.  I upgraded the g4 bot to 4.5.20, since that didn't have any version of bsddb on it.  I also installed 4.4.20 on the amd64 buildbot.  The x86 gentoo bot probably has 4.1, but it's down right now.  Martin's machine has 4.2.

Both versions (4.4 and 4.5) require the flag DB_DBT_MALLOC (get() returns an error 22 otherwise).  However only 4.4 actually allocates memory.  4.5 doesn't.  I checked in revs 55457 (2.5) which should handle this better.  Could someone please test and verify I really fixed the problem this time.  If it's really fixed, close this bug report.

The real change is to conditionally free data if it is a different pointer than we pass in (orig_data).  Don't free if they are the same pointer.

Tested 4.5 memory leaks with the leaks program and 4.4 with valgrind.
File Added: db.c

File Added: test_get_both.py

Seems to be really fixed now.
55457 (2.5)

Committed revision 55751. (2.6)