New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refleak tests: test_doctest and test_gc are failing #45881
Comments
I've seen the problem on Windows only. test_doctest fails and the File "c:\dev\python\py3k\lib\test\test_doctest.py", line 1570, in
test.test_doct
est.test_debug
Failed example:
try: doctest.debug_src(s)
finally: sys.stdin = real_stdin
Expected:
> <string>(1)<module>()
(Pdb) next
12
--Return--
> <string>(1)<module>()->None
(Pdb) print(x)
12
(Pdb) continue
Got:
> c:\dev\python\py3k\lib\io.py(281)__del__()
-> try:
(Pdb) next
> c:\dev\python\py3k\lib\io.py(282)__del__()
-> self.close()
(Pdb) print(x)
*** NameError: NameError("name 'x' is not defined",)
(Pdb) continue
12
**********************************************************************
1 items had failures:
1 of 4 in test.test_doctest.test_debug
***Test Failed*** 1 failures.
test test_doctest failed -- 1 of 418 doctests failed
test_gc
test test_gc failed -- Traceback (most recent call last):
File "c:\dev\python\py3k\lib\test\test_gc.py", line 193, in test_saveall
self.assertEqual(gc.garbage, [])
AssertionError: [<io.BytesIO object at 0x01237968>] != [] 2 tests failed: |
After some hard debugging:
Some possible directions:
|
Finally I found a potential problem with the garbage collector in a
Now, try to follow my explanation (if I correctly understand the gc
I join a script which reproduces the behaviour. Note that 2.4 and 2.5 |
Hoping to draw Tim into this... He's the only one I know who truly |
I'm trying to figure out the attached script. If I run Python 3.0, the script doesn't run because of the undefined gc.DEBUG_OBJECTS. If I just remove that, the script runs without error. Does that mean the problem is fixed? Or is running without an error an example of the problem? If I add gc.DEBUG_SAVEALL, it fails--but that seems obvious because DEBUG_SAVEALL adds all objects with finalizers to gc.garbage. |
The attached script does not "fail". The weird thing is that after gc.collect(), gc.garbage is not empty. |
I spent some time to understand the example script today. The specific issue is that a set of objects get put into the list of unreachable objects with finalizers (both Immutable and Finalizer instances). When Cycle's __dict__ is cleared, it also decrefs Immutable which resurrects it and Finalizer. The garbage collector is not prepared for an unreachable finalizer object to become reachable again. More generally, it's hard to assume anything about the state of the finalizers after unreachable trash is collected. I'll think more about what to do, but I don't see any easy solutions. |
One last thought on this bug. The problem is that after we try to delete garbage, we really can't know much about the state of the objects in the finalizers list. If any of the objects that are cleared end up causing a finalizer to run, then any of the objects in the finalizers list may be reachable again. One possibility is to do nothing with the objects in the finalizers list if there was any garbage to delete. That means objects with finalizers would be harder to get to gc.collect()--for example, you'd need to call gc.collect() twice in a row. The first time to clear garbage, the second time to handle unreachable objects with finalizers. Or the GC could run a second time if garbage was cleared and finalizers was non-empty. A more complicated possibility would be to track some object state about when a finalizer was run. If any of the objects in finalizers had a finalizer that ran while garbage was cleared, we could skip the finalizers list. I don't know how to implement this, since an arbitrary C type could run Python code in tp_dealloc without notifying GC. |
What if we simply run another collection on the finalizers list? |
Amaury-- I think that will work. I put together a small patch that seems to pass all the tests, but it too messy. We need some care to make sure we don't spin forever if there's some degenerate case where we never escape GC. |
The code is still in no shape to submit. It has lots of debugging prints in it, etc. but the basic structure might work. Do you want to let me know if it makes sense? |
Can we have an update on this please as it seems important. |
ping :) |
Is the error still current? io.StringIO is now completely implemented in _io/textio.c, and should not have any Python-level __del__. |
I don't know ... Is somebody able to test it? |
The GC behaves gracefully in 3.4: the gc_bug script shows no uncollectable object. I don't think this is worth fixing in 3.3. |
I agree with not fixing 3.3. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: