Patch proposed to fix the refleaks.

Patch updated for latest trunk.

Fixed with r78563 and r78564.

Florent, Could you explain the changes to the unittest?. I don't understand them.

> Could you explain the changes to the unittest?

The reference to "self" in the hooks were preventing the GC of the test case, as far as I understand, because it creates a cycle.
When using weak references, there's no more dead cycles.
It is my own explanation, maybe it's not exactly what happens.

For the sys.traceback, I could not explain it better than the FAQ:
http://docs.python.org/faq/design.html#how-does-python-manage-memory

I may add a small comment in test_replication to explain the usage of weakref.

I confirm the leaks in the C code. How do you find them?

I see your point with the weakrefs in the unittest, but I don't see why they are so important. We are creating a GC cycle, yes, but the GC will collect it eventually, beside the reference counters being fooled because the cycle. Are you using a tool to find cycles?

I am committing your patch upstream and I need to understand the rationale :).

Thanks!.

The tool is called regrtest:

~ $ ./python -m test.regrtest -R 2:3: -uall test_bsddb3

Some devs and some buildbots hunt refleaks in the test suite.
Even if the test-related refleaks are not important, they may hide real refleaks in the source code.

You need a debug build of Python to use -R:
./configure --with-pydebug && make

See also: http://www.python.org/dev/faq/#how-do-i-create-a-debug-build-of-python

> I confirm the leaks in the C code. How do you find them?

I've done it manually, disabling some tests, and running "regrtest -R" repeatedly until I isolate the function which is responsible.
Then I studied the source code, looking for the missing DECREFs.

I have spend a few hours trying to understand the issue deeply, and I have an easier to understand version. It is bigger, but explain the issue good enough to use in a production program.

Please, Florent, review. It pass the leak test.

I rather prefer this patch because the weak references were sort of magic, and this new code actually shows what you should use in production.

If you think the patch is OK, it will be integrated in pybsddb 4.8.4 and then backported for python 2.7.

I also solved an undesired coupling between Replication Manager and Base Replication testsuites.

Jesús, your patch looks good. I prefer your approach.
My use of the weakrefs was an ugly workaround, I admit.

Maybe the "close()" method in the C module should be in charge of DECREFing the handlers and freeing memory. I did not look further in this direction.

Attached patch is quite the same as yours. I just simplified the "dummy()" handler.

Patch up-ported to pybsddb 4.8.4.

I have problem getting your changes working in python 3.x. Yes, pybsddb supports py3k.

The problem is line "sys.exc_traceback = sys.last_traceback = None", that throws the following error under python 3.x:

"""
File "/home/pybsddb/build/lib.solaris-2.10-i86pc-3.1/bsddb3/tests/test_compare.py", line 215, in verifyStderr
    sys.exc_info()[2] = sys.last_traceback = None
TypeError: 'tuple' object does not support item assignment
"""

The line is automatically translated by "2to3".

Any suggestion?.

I guess in 3.x we can change the line to "sys.exc_clear()", and forget about the "None" asignment. What do you think?.