Issue1540
Created on 2007-12-02 14:41 by christian.heimes, last changed 2008-01-06 22:29 by admin.
| File name |
Uploaded |
Description |
Edit |
Remove |
|
gc_bug.py
|
amaury.forgeotdarc,
2007-12-05 16:51
|
|
|
|
|
msg58089 - (view) |
Author: Christian Heimes (christian.heimes) |
Date: 2007-12-02 14:41 |
|
I've seen the problem on Windows only. test_doctest fails and the
problem also causes test_gc to fail when it is run after test_doctest.
W/o a prior run of test_doctest test_gc doesn't fail.
File "c:\dev\python\py3k\lib\test\test_doctest.py", line 1570, in
test.test_doct
est.test_debug
Failed example:
try: doctest.debug_src(s)
finally: sys.stdin = real_stdin
Expected:
> <string>(1)<module>()
(Pdb) next
12
--Return--
> <string>(1)<module>()->None
(Pdb) print(x)
12
(Pdb) continue
Got:
> c:\dev\python\py3k\lib\io.py(281)__del__()
-> try:
(Pdb) next
> c:\dev\python\py3k\lib\io.py(282)__del__()
-> self.close()
(Pdb) print(x)
*** NameError: NameError("name 'x' is not defined",)
(Pdb) continue
12
**********************************************************************
1 items had failures:
1 of 4 in test.test_doctest.test_debug
***Test Failed*** 1 failures.
test test_doctest failed -- 1 of 418 doctests failed
test_gc
test test_gc failed -- Traceback (most recent call last):
File "c:\dev\python\py3k\lib\test\test_gc.py", line 193, in test_saveall
self.assertEqual(gc.garbage, [])
AssertionError: [<io.BytesIO object at 0x01237968>] != []
2 tests failed:
test_doctest test_gc
|
|
msg58205 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) |
Date: 2007-12-05 00:34 |
|
After some hard debugging:
- doctest.debug_src() is unlucky enough to trigger a garbage collection
just when compiling the given code.
- gc collects unreachable objects, among them is an instance of the
class doctest._SpoofOut, which derives from io.StringIO.
- The debugger steps into io.IOBase.__del__
Some possible directions:
- Change the gc thresholds. A very temporary workaround to make the test
pass.
- Find the cycle involving the SpoofOut object, and try to break it in
doctest.py.
- Find a way to disable pdb tracing when the gc is running finalizers.
(this is what happens in 2.5: pdb does not step into a C function)
- Forget everything, and wait for the io.py object to be rewritten in C.
|
|
msg58218 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) |
Date: 2007-12-05 16:51 |
|
Finally I found a potential problem with the garbage collector in a
specific case:
- Some object C participates in a reference cycle, and contains a
reference to another object X.
- X.__del__ 'resurrect' the object, by saving its 'self' somewhere else.
- X contains a reference to Y, which has a __del__ method.
When collecting all this, gc.garbage == [Y] !
This is not true garbage: if you clean gc.garbage, then the next
gc.collect() clears everything.
Now, try to follow my explanation (if I correctly understand the gc
internals):
- When the cycle is garbage collected, X and Y are detected as
'unreachable with finalizers', and put in a specific 'finalizers' list.
- the cycle is broken, C is deallocated.
- This correctly triggers X.__del__. X is removed from the 'finalizers'
list.
- when X is resurrected, it comes back to the normal gc tracking list.
- At the end, 'finalizers' contains 3 objects: X.__dict__, Y and Y.__dict__.
- Y is then considered as garbage.
I join a script which reproduces the behaviour. Note that 2.4 and 2.5
are affected too.
In py3k, the 'resurrect' seems to be caused by a (caught) exception in
TextIOWrapper.close(): the exception contains a reference to the frame,
which references the self variable.
|
|
msg58221 - (view) |
Author: Guido van Rossum (gvanrossum) |
Date: 2007-12-05 17:25 |
|
Hoping to draw Tim into this... He's the only one I know who truly
understands these issues...
|
|
| Date |
User |
Action |
Args |
| 2008-01-06 22:29:44 | admin | set | keywords:
- py3k versions:
Python 3.0 |
| 2007-12-05 17:25:27 | gvanrossum | set | assignee: tim_one messages:
+ msg58221 nosy:
+ tim_one |
| 2007-12-05 16:51:42 | amaury.forgeotdarc | set | files:
+ gc_bug.py nosy:
+ gvanrossum messages:
+ msg58218 components:
+ Interpreter Core |
| 2007-12-05 00:34:04 | amaury.forgeotdarc | set | nosy:
+ amaury.forgeotdarc messages:
+ msg58205 |
| 2007-12-02 14:41:56 | christian.heimes | create | |
|