classification
Title: tests failing when run under coverage
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: brett.cannon Nosy List: Kristian.Vlaardingerbroek, brett.cannon
Priority: normal Keywords: patch

Created on 2011-01-24 02:27 by brett.cannon, last changed 2011-02-22 03:04 by brett.cannon. This issue is now closed.

Files
File name Uploaded Description Edit
test_gc.diff brett.cannon, 2011-01-28 05:37 test_gc fix
test_descr.diff brett.cannon, 2011-01-28 05:39 test_descr fix
issue10992.diff brett.cannon, 2011-01-28 21:38 Fix all failing tests when run under coverage.py
Messages (9)
msg126913 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2011-01-24 02:27
A bunch of tests fail when run under coverage (trend seems to be refcount tests). This is to act as a meta-issue to keep track of what tests need to be fixed.

[fail under both coverage.py and regrtest -T]
test_ctypes
test_descr
test_gc
test_metaclass
test_pydoc
test_trace

[coverage.py only]
test_genexps
test_sys

[regrtest -T only]
test_array
test_importlib
test_runpy
msg127019 - (view) Author: Kristian Vlaardingerbroek (Kristian.Vlaardingerbroek) Date: 2011-01-25 16:53
Following tests in test_descr.py fail with both methods:

test_iter_items
test_iter_keys
test_iter_values

regrtest -T causes the following test to fail aswell:

test_slots

The test_iter_* tests fail because __locals__ gets added to the namespace of the class it is testing when tracing is enabled.

test_slots fails on line 1031 in a section marked "Test lookup leaks [SF bug 572567]"
msg127022 - (view) Author: Kristian Vlaardingerbroek (Kristian.Vlaardingerbroek) Date: 2011-01-25 17:15
Following tests in test_descr.py fail:

test_collect_generations (line 261)
test_frame (line 183)
test_get_count (line 249)

These tests count the number of allocations and a trace function can mess this up.
msg127030 - (view) Author: Kristian Vlaardingerbroek (Kristian.Vlaardingerbroek) Date: 2011-01-25 18:09
test_metaclass has some doctests failing because of the added __locals__.
msg127031 - (view) Author: Kristian Vlaardingerbroek (Kristian.Vlaardingerbroek) Date: 2011-01-25 18:12
msg127022 applies to test_gc and not test_descr
msg127246 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2011-01-28 05:37
Attached is a patch against test_gc that shows what I think should be done for the tests failing because of refcounts. Basically I added a refcount_test decorator to test.support that skips a decorated test if it is not being run under CPython and otherwise unsets the trace function.

This should work for all tests that are discovered to fail because of refcounts.
msg127248 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2011-01-28 05:39
Attached is a patch for test_descr. I suspect the __locals__ failures will all be the same; simply skip the test when a trace function is set.

And thanks for identifying the cause of the failures, Kristian; helps me make sure I am not misdiagnosing the problem.
msg127352 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2011-01-28 21:38
The attached patch has all tests passing when run under coverage.py. Most places are flat-out skipped since there is no good way around the tests short of coding up a second set of test values. Plus editing doctests to have optional values is way too painful, so in those cases I just skip the whole doctest.
msg129028 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2011-02-22 03:04
3.3 in r88494
History
Date User Action Args
2011-02-22 03:04:39brett.cannonsetstatus: open -> closed

messages: + msg129028
resolution: fixed
stage: needs patch -> resolved
2011-01-31 19:48:28brett.cannonsetassignee: brett.cannon
2011-01-28 21:38:48brett.cannonsetfiles: + issue10992.diff

messages: + msg127352
2011-01-28 05:39:11brett.cannonsetfiles: + test_descr.diff

messages: + msg127248
2011-01-28 05:37:03brett.cannonsetfiles: + test_gc.diff

messages: + msg127246
keywords: + patch
2011-01-25 18:12:12Kristian.Vlaardingerbroeksetmessages: + msg127031
2011-01-25 18:09:59Kristian.Vlaardingerbroeksetmessages: + msg127030
2011-01-25 17:15:57Kristian.Vlaardingerbroeksetmessages: + msg127022
2011-01-25 16:53:13Kristian.Vlaardingerbroeksetmessages: + msg127019
2011-01-24 10:28:47Kristian.Vlaardingerbroeksetnosy: + Kristian.Vlaardingerbroek
2011-01-24 02:27:04brett.cannoncreate