Issue967763
Created on 2004-06-06 20:51 by nnorwitz, last changed 2004-12-20 00:00 by gregory.p.smith.
| File name |
Uploaded |
Description |
Edit |
Remove |
|
bsddb.patch
|
nnorwitz,
2004-06-06 20:51
|
mem leak patch 1 |
|
|
|
sf-patch-967763-fix.patch
|
gregory.p.smith,
2004-06-27 23:34
|
committed patch for related memleaks |
|
|
|
msg46122 - (view) |
Author: Neal Norwitz (nnorwitz) |
Date: 2004-06-06 20:51 |
|
only one of the changes fixed a leak reported by
valgrind, but i think all the changes are necessary.
|
|
msg46123 - (view) |
Author: Neal Norwitz (nnorwitz) |
Date: 2004-06-13 20:56 |
|
Logged In: YES
user_id=33168
Greg, just wanted to make sure you saw this. It seems like
you're the only one working on bsddb these days.
|
|
msg46124 - (view) |
Author: alan johnson (chomo) |
Date: 2004-06-14 21:11 |
|
Logged In: YES
user_id=943591
looks like the ones needed
anyway
|
|
msg46125 - (view) |
Author: Gregory P. Smith (gregory.p.smith) |
Date: 2004-06-15 00:02 |
|
Logged In: YES
user_id=413
make_key_dbt() will never return with an error leaving a
FREE_DBT(key) necessary.
however looking in depth at the code i believe it would have
been possible to have a leak when using integer keys and
exiting early with an error. a patch (untested) that should
fix that is attached. i'll test it out later.
do you have any memleak test cases by any chance?
|
|
msg46126 - (view) |
Author: Neal Norwitz (nnorwitz) |
Date: 2004-06-15 01:36 |
|
Logged In: YES
user_id=33168
I just run the regression tests. Did you see my mail to
python-dev?
http://mail.python.org/pipermail/python-dev/2004-June/045245.html
Here's the bsddb leaks reported by valgrind:
4 bytes in 1 blocks are definitely lost in loss record 7 of 663
at 0x3C01DA1D: malloc (vg_replace_malloc.c:109)
by 0x3C928265: make_key_dbt (_bsddb.c:400)
by 0x3C92DE4F: DBC_set_range (_bsddb.c:2911)
8 bytes in 2 blocks are definitely lost in loss record 29
of 663
at 0x3C01DA1D: malloc (vg_replace_malloc.c:109)
by 0x3C928265: make_key_dbt (_bsddb.c:400)
by 0x3C929088: DB_get (_bsddb.c:1349)
|
|
msg46127 - (view) |
Author: Gregory P. Smith (gregory.p.smith) |
Date: 2004-06-15 01:47 |
|
Logged In: YES
user_id=413
Thanks, I hadn't caught up on python-dev in a while. What
happens if you apply my attached patch and rerun the test
under valgrind?
The 4 bytes leaked sounds exactly like what i was expecting
reading the code (an integer key causes make_key_dbt to
alloc a 32bit value). in the DBC_set_range case without the
patch i believe it is likely that the integer is allocated
but the REALLOC flag is overridden in the old code with a
MALLOC causing BerkeleyDB to leak the old pointer and
allocate a new key for the return value (btree accessed
using an integer key).
8 bytes in 2 blocks leaked in DB_get is double what i
expected but without debug info showing the DB_get function
call arguments and database flags its hard to say what happened.
|
|
msg46128 - (view) |
Author: Neal Norwitz (nnorwitz) |
Date: 2004-06-16 01:48 |
|
Logged In: YES
user_id=33168
should FREE_DBT() be called clearTxn() and friends fail?
one memory leak is caused in DB_get() by test_recno, line 75
(d.get(100)). if the FREE_DBT()s at line 1392-1393 are
moved outside the if, so they always execute the leak goes
away. i'm not sure if this is really the correct solution
though.
the other leak is in test_compat, don't know anything more.
|
|
msg46129 - (view) |
Author: Gregory P. Smith (gregory.p.smith) |
Date: 2004-06-27 22:51 |
|
Logged In: YES
user_id=413
DB_get and DBC_set_range were the two leaks that showed up in valgrind using the existing testsuite. however there were plenty more possible leak cases (most of them in odd situations like out of memory or passing a non DBTxn object in the txn parameter).
The most common trigger for a leak in various places would be an exception exit from a many functions when using integer keys (queue & recno databases). I have a patch which i believe fixes all of these and more after a lot of code inspection that i'll commit once my valgrind run finishes and reveals no bsddb related leaks or double frees. (ugh running valgrind -v --leak-check=yes ./python Lib/bsddb/test/test_all.py on a 5 year old laptop is slow...)
"manual memory management... how quaint."
|
|
msg46130 - (view) |
Author: Gregory P. Smith (gregory.p.smith) |
Date: 2004-06-27 23:34 |
|
Logged In: YES
user_id=413
fix committed. Modules/_bsddb.c 1.32 and Lib/bsddb/test/test_recno.py 1.8
patch attached. marking as python 2.3 as this should be committed to the python 2.3 maintenance branch before 2.3.5 is released.
|
|
msg46131 - (view) |
Author: Gregory P. Smith (gregory.p.smith) |
Date: 2004-12-20 00:00 |
|
Logged In: YES
user_id=413
fixed, committed to the release23-maint branch.
|
|
| Date |
User |
Action |
Args |
| 2004-06-06 20:51:04 | nnorwitz | create | |
|