Created on 2006-02-11 20:22 by mgedmin, last changed 2007-11-23 17:10 by skip.montanaro. This issue is now closed.
|py2.4-trace-fix-doctests.patch||mgedmin, 2006-02-11 20:22||patch|
|msg49483 - (view)||Author: Marius Gedminas (mgedmin) *||Date: 2006-02-11 20:22|
SchoolTool's test runner uses the 'trace' module to produce unit test coverage reports. Many unit tests in SchoolTool are actually doctests. When the test runner invokes trace.CoverageResults.write_results, it gets a lot of printed error messages like this: Not printing coverage data for '<doctest schooltool.app.browser.tests.test_app.doctest_ACLView>': [Errno 2] No such file or directory: '<doctest schooltool.app.browser.tests.test_app.doctest_ACLView>' and a lot of empty files with nimes like <doctest schooltool.timetable.browser.cover appear in the output directory. The attached patch fixes both problems.
|msg49484 - (view)||Author: Jim Jewett (jimjjewett)||Date: 2006-02-15 16:20|
Logged In: YES user_id=764593 Do I understand this correctly, then (1) doctests are still run, and contribute to coverage (2) The code that creates a test should not be measured for coverage. ??? (3) Because doctests are in the same file as the code, rather than the other tests, they get mistaken for code whose own coverage should be measured. If so, it seems like the right solution is to add a subdirectory for test-portions-that-didn't-run, and put doctests there instead of in the main directory. Or are you saying that is OK if some of the tests (and some of the doctests) don't actually run in full? -jJ
|msg49485 - (view)||Author: Marius Gedminas (mgedmin) *||Date: 2006-02-15 17:06|
Logged In: YES user_id=44660 I do not entirely understand your questions, so I'll try to describe the situation more clearly. Imagine this code in foo.py: import doctest def do_something(x): """ >>> do_something(3) 4 """ if x == 0: return -1 return x + 1 if __name__ == '__main__': import doctest doctest.testmod() Now run python -m trace --cover --missing --coverdir=output foo.py trace.py will produce 'foo.cover' that looks like this 1: import doctest 1: def do_something(x): """ >>> do_something(3) 4 """ 1: if x == 0: >>>>>> return -1 1: return x + 1 1: if __name__ == '__main__': 1: import doctest 1: doctest.testmod() It will also produce an empty file called '<doctest __main__.cover', and produce a warning on stdout Not printing coverage data for '<doctest __main__.do_something>': [Errno 2] No such file or directory: '<doctest __main__.do_something>' With my patch, trace.py will produce a 'foo.cover' file with the same content, and it will not create empty files nor produce any warnings. (I do not particularly care about tracing Python statements inside doctests themselves. I would like to keep this bug report focused, so if anyone wants doctest coverage in addition to regular Python code coverage, I suggest opening a new bug report.) Given this, I think the answers to your questions are (1) yes (2) what is 'the code that creates a test'? If you mean the '>>> do_something(4)' line, then I'm content if it remains untracked (3) whether doctests are in the same file or not is irrelevant, in my opinion (in SchoolTool most of the doctests are in separate files); I do not see where they might get mistaken for code There are no test-portions-that-didn't run, so there's no need for subdirectories. It would not be OK if some of the tests/doctests didn't actually run because of trace.py. Luckily, that is not the case.
|msg49486 - (view)||Author: STINNER Victor (haypo) *||Date: 2007-02-27 01:45|
Zope version of doctest is patched for coverage testing, get: http://svn.zope.org/zope.testing/trunk/src/zope/testing/doctest.py The patch to fix Python 2.4 doctest.py: --- doctest.py.orig 2007-02-27 02:32:05.000000000 +0100 +++ doctest.py 2007-02-27 02:36:31.000000000 +0100 @@ -340,8 +340,19 @@ """ def __init__(self, out): self.__out = out + self.__debugger_used = False pdb.Pdb.__init__(self) + def set_trace(self): + self.__debugger_used = True + pdb.Pdb.set_trace(self) + + def set_continue(self): + # Calling set_continue unconditionally would break unit test coverage + # reporting, as Bdb.set_continue calls sys.settrace(None). + if self.__debugger_used: + pdb.Pdb.set_continue(self) + def trace_dispatch(self, *args): # Redirect stdout to the given stream. save_stdout = sys.stdout
|msg57792 - (view)||Author: Skip Montanaro (skip.montanaro) *||Date: 2007-11-23 17:10|
|2007-11-23 17:10:37||skip.montanaro||set||status: open -> closed|
messages: + msg57792
nosy: + skip.montanaro