New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
reference leaks in pdb #64965
Comments
After the pdb 'continue' command, the signal module owns a reference to Pdb.sigint_handler. On the next instantiation of pdb, the signal module owns a reference to a new sigint_handler method that owns a reference to the previous sigint_handler. As a consequence, the first pdb instance is never freed (as well as the frames that are referenced by this pdb instance). The following test demonstrates the problem that is fixed by the attached patch. Run the following script: # START of refleak.py
import sys, gc, pdb
i = 1
while i:
pdb.set_trace()
x = 1
gc.collect(); print(sys.gettotalrefcount()) # END of refleak.py With the following pdb commands: $ python refleak.py
> /tmp/test/refleak.py(6)<module>()
-> x = 1
(Pdb) continue
95898
> /home/xavier/tmp/test/refleak.py(5)<module>()
-> pdb.set_trace()
(Pdb) continue
95983
> /tmp/test/refleak.py(6)<module>()
-> x = 1
(Pdb) continue
96068
> /tmp/test/refleak.py(5)<module>()
-> pdb.set_trace()
(Pdb) i = 0
(Pdb) continue
96153 |
The first pdb instance is (and all the other pdb instances) never freed until the call to PyOS_FiniInterrupts() in Py_Finalize(). |
After applying patch 'regrtest.diff' from bpo-20746, the command: $ ./python -m test -W -R3:3 test_pdb succeeds now and shows there are no reference leaks. However, with the following change in Lib/test/test_pdb.py and patch 'regrtest.diff' applied: diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py
--- a/Lib/test/test_pdb.py
+++ b/Lib/test/test_pdb.py
@@ -34,7 +34,7 @@
"""This tests the custom displayhook for pdb.
>>> def test_function(foo, bar):
- ... import pdb; pdb.Pdb(nosigint=True).set_trace()
+ ... import pdb; pdb.Pdb().set_trace()
... pass
>>> with PdbTestInput([ there are reference leaks: $ ./python -m test -R 3:3 test_pdb
[1/1] test_pdb
beginning 6 repetitions
123456
......
test_pdb leaked [200, 200, 200] references, sum=600
test_pdb leaked [43, 43, 43] memory blocks, sum=129
1 test failed:
test_pdb And now with the proposed patch in the previous msg 212171 'refleak.patch' of this issue that fixes the sigint_handler leak, and also patch 'regrtest.diff' applied: $ ./python -m test -R 3:3 test_pdb
[1/1] test_pdb
beginning 6 repetitions
123456
......
1 test OK. |
An improved patch with a test case. |
It's not easy to test the patch because running test_pdb fails with the following error (I also get the error if the patch is not applied): $ ./python -m test test_pdb test_pdb
[1/2] test_pdb
[2/2] test_pdb
test test_pdb failed -- Traceback (most recent call last):
File "/home/haypo/prog/python/default/Lib/doctest.py", line 2189, in runTest
test, out=new.write, clear_globs=False)
AssertionError: Failed doctest test for test.test_pdb.test_next_until_return_at_return_event
File "/home/haypo/prog/python/default/Lib/test/test_pdb.py", line 603, in test_next_until_return_at_return_event File "/home/haypo/prog/python/default/Lib/test/test_pdb.py", line 617, in test.test_pdb.test_next_until_return_at_return_event |
See bpo-20746 for the test_pdb failure when run multiple times. |
This is because breakpoints number are class attributes. With the following change, the "./python -m test test_pdb test_pdb" is ok: $ hg diff
diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py
--- a/Lib/test/test_pdb.py
+++ b/Lib/test/test_pdb.py
@@ -614,6 +614,8 @@
... test_function_2()
... end = 1
+ >>> from bdb import Breakpoint; Breakpoint.next = 1
+
>>> with PdbTestInput(['break test_function_2',
... 'continue',
... 'return', Attached refleak_3.patch fixes this. |
After refleak_3.patch, test_pdb_next_command_in_generator_for_loop is the only remaining test that sets a breakpoint without reinitializing the breakpoints numbering first. |
Will the issue be fixed in Python formal build? I'm using pdb to debug a Python script, there are global variables in the script. Duo the leak of sigint_handler, calling gc.collect() doesn't release the global variables until calling PyOS_FiniInterrupts(). |
Now, I have to set nosigint to True to fix the reference leaks issue for my app. |
The problem may still be reproduced by running the attached pdb_refleak.py script. Enter three times 'c' (short for the pdb 'continue' command) and the following line is printed:
This shows that the third pdb instance 'pdb 3', owns indirectly a reference to the second and first pdb instances. As the _signal extension module owns a reference to 'pdb 3' through its sigint_handler() method, then it also owns indirectly a reference to 'pdb 1'. |
Oh, test_pdb does not fail anymore in refleak mode as expected (see msg212510 for the change to apply to test_pdb.py in order to reproduce that). 'hg bisect' says that this is happening since:
And indeed, this changeset has removed entirely the doctests from the test suite :( |
Entered new bpo-28338. |
Uploaded refleak_5.patch with a simpler test case. The '_previous_sigint_handler' is reset at the Pdb prompt - instead of when the signal is triggered - to handle the case where pdb stops at a new set_trace() hard breakpoint after it has been run with 'continue'. As a side effect, this also fixes bpo-22502. |
New changeset 31a2d270c0c3 by Xavier de Gaye in branch '3.5': New changeset 86a1905ea28d by Xavier de Gaye in branch '3.6': New changeset 8bb426d386a5 by Xavier de Gaye in branch 'default': |
Good to know the fix will be included Python official builds. Thanks. |
Please see https://bugs.python.org/issue36667 for a followup. It does not look like moving it to I think it should be restored in both places. |
Misc/NEWS
so that it is managed by towncrier #552Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: