New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
assertRaises as a context manager keeps tracebacks and frames alive #54024
Comments
I noticed regrtest claimed it cannot delete folder after test_tarfile Traceback (most recent call last):
File "e:\python-dev\py3k\lib\runpy.py", line 160, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "e:\python-dev\py3k\lib\runpy.py", line 73, in _run_code
exec(code, run_globals)
File "e:\python-dev\py3k\lib\test\test_tarfile.py", line 1566, in <module>
test_main()
File "e:\python-dev\py3k\lib\test\test_tarfile.py", line 1563, in test_main
shutil.rmtree(TEMPDIR)
File "e:\python-dev\py3k\lib\shutil.py", line 283, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File "e:\python-dev\py3k\lib\shutil.py", line 281, in rmtree
os.remove(fullname)
WindowsError: [Error 32] プロセスはファイルにアクセスできません。別のプロセスが
使用中です。: 'E:\\PYTHON~1\\py3k\\@test_5276_tmp\\tmp.tar' # It says "Process cannot access the file. Another process is using it." I tried to reproduce this by running test_tarfile alone, but failed. (Probably there is timing problem like GC... The reason why I think I think _Stream object left opened after exception occured in (even |
Just to be sure: Have you checked this bug doesn’t apply to 2.7? Can you add a regression test? |
I created separate small test case to reproduce, and I tried it on And exception might not be related. Because error still occurred But if I removed "#" before gc.collect(), test case runs fine. And I copied Python3.1's Lib/tarfile.py into Python3.2's Lib, |
I think the tests should go into 2.7 and 3.1 even if they don’t have the bug. Adding Antoine to nosy, since he’s listed for gc in Misc/maintainers.rst. |
I tried test_tar_pipe_open_read_error_v2.py on py3k, I didn't see uncollectable objects in release27-maint And I replaced py3k's Lib/unittest with release31-maint's E:\python-dev>py3k test_tar_pipe_open_read_error_v2.py OK Traceback (most recent call last):
File "test_tar_pipe_open_read_error_v2.py", line 25, in test_main
unittest.main()
File "e:\python-dev\py3k\lib\unittest\main.py", line 95, in __init__
self.runTests()
File "e:\python-dev\py3k\lib\unittest\main.py", line 231, in runTests
sys.exit(not self.result.wasSuccessful())
SystemExit: False
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "test_tar_pipe_open_read_error_v2.py", line 33, in <module>
test_main()
File "test_tar_pipe_open_read_error_v2.py", line 30, in test_main
shutil.rmtree(TEMPDIR)
File "e:\python-dev\py3k\lib\shutil.py", line 283, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File "e:\python-dev\py3k\lib\shutil.py", line 281, in rmtree
os.remove(fullname)
WindowsError: [Error 32] プロセスはファイルにアクセスできません。別のプロセスが
使用中です。: 'c:\\docume~1\\ocean\\locals~1\\temp\\__foobarbaz__\\tmp.tar'
gc: 61 uncollectable objects at shutdown:
[54634 refs] |
The error went away when I commented out following line. Lib/unittest/case.py(133) I found this by brute force.... I noticed that And when the exception was re-raised like this, (_Stream#_read)
IOError will be set to ReadError's *cause* attribute. # gc.get_referents() was really helpful. I'll attach the test script "test_assert_raises.py" to reproduce E:\python-dev>py3k test_assert_raises.py OK |
Here is the patch to fix this issue. (Please forget first patch) E:\python-dev>py3k -m test.regrtest test_tarfile E:\python-dev>py3k test_assert_raises.py OK |
You shouldn't use DEBUG_LEAK except for debugging purposes :) As the doc mentions: “To debug a leaking program call gc.set_debug(gc.DEBUG_LEAK). Notice that this includes gc.DEBUG_SAVEALL, causing garbage-collected objects to be saved in gc.garbage for inspection.” It's true that the message at shutdown should be eliminated in this case, though. |
Here is the simple test case to demonstrate this issue. |
Adding Michael for review of py3k_fix__AssertRaisesContext.patch. |
Note that the original issue (test_tarfile failures on the Windows buildbots) now seems fixed thanks to the various tarfile and test_tarfile improvements. I have no opinion on whether assertRaises should take care of cleaning the tracebacks or not. As the test_tarfile issue shows, the resource consumption issues were tied to actual bugs both in the library and in the test suite. |
Yes, thanks. :-) |
The patch py3k_fix__AssertRaisesContext.patch looks good. A test would be nice. The code already attempts to sanitize the traceback, so sanitizing __cause__ and __context__ seems reasonable. |
frame.clear() was committed in bpo-17934, it would allow a less brutal resolution. |
See bpo-1565525 for the new helper function in the traceback module. |
New changeset 6ab3193e890e by Antoine Pitrou in branch '3.4': New changeset 553fe27521be by Antoine Pitrou in branch 'default': |
I've committed an alternate patch using traceback.clear_frames(). |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: