Author Yaroslav.Halchenko
Recipients Yaroslav.Halchenko, brian.curtin, exarkun, fperez, michael.foord, pitrou
Date 2010-04-09.01:53:43
SpamBayes Score 1.85049e-08
Marked as misclassified No
Message-id <1270778026.43.0.929569734308.issue7897@psf.upfronthosting.co.za>
In-reply-to
Content
In PyMVPA we have our little decorator as an alternative to Fernando's generators,  and which is closer, I think, to what Michael was wishing for:
@sweepargs

http://github.com/yarikoptic/PyMVPA/blob/master/mvpa/testing/sweepargs.py

NB it has some minor PyMVPA specificity which could be easily wiped out, and since it was at most 4 eyes looking at it and it bears "evolutionary" changes, it is far from being the cleanest/best piece of code, BUT:

* it is very easy to use, just decorate a test method/function and give an argument which to vary within the function call, e.g smth like

@sweepargs(arg=range(5))
def test_sweepargs_demo(arg):
    ok_(arg < 5)
    ok_(arg < 3)
    ok_(arg < 2)

For nose/unittest it would still look like a single test

* if failures occur, sweepargs groups failures by the type/location of the failures and spits out a backtrace for one of failures + summary (instead of detailed backtraces for each failure) specifying which arguments lead to what error... here is the output for example above:

$> nosetests -s test_sweepargs_demo.py
F
======================================================================
FAIL: mvpa.tests.test_sweepargs_demo.test_sweepargs_demo
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.5/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "/usr/lib/pymodules/python2.5/nose/util.py", line 630, in newfunc
    return func(*arg, **kw)
  File "/home/yoh/proj/pymvpa/pymvpa/mvpa/tests/test_sweepargs_demo.py", line 11, in test_sweepargs_demo
    ok_(arg < 2)
  File "/usr/lib/pymodules/python2.5/nose/tools.py", line 25, in ok_
    assert expr, msg
AssertionError: 
 Different scenarios lead to failures of unittest test_sweepargs_demo (specific tracebacks are below):
  File "/home/yoh/proj/pymvpa/pymvpa/mvpa/tests/test_sweepargs_demo.py", line 10, in test_sweepargs_demo
    ok_(arg < 3)
    File "/usr/lib/pymodules/python2.5/nose/tools.py", line 25, in ok_
    assert expr, msg
  on
    arg=3 
    arg=4 

  File "/home/yoh/proj/pymvpa/pymvpa/mvpa/tests/test_sweepargs_demo.py", line 11, in test_sweepargs_demo
    ok_(arg < 2)
    File "/usr/lib/pymodules/python2.5/nose/tools.py", line 25, in ok_
    assert expr, msg
  on
    arg=2 

----------------------------------------------------------------------
Ran 1 test in 0.003s

FAILED (failures=1)

* obviousely multiple decorators could be attached to the same test, to test on all combinations of more than 1 argument but output atm is a bit cryptic ;-)
History
Date User Action Args
2010-04-09 01:53:46Yaroslav.Halchenkosetrecipients: + Yaroslav.Halchenko, exarkun, pitrou, michael.foord, brian.curtin, fperez
2010-04-09 01:53:46Yaroslav.Halchenkosetmessageid: <1270778026.43.0.929569734308.issue7897@psf.upfronthosting.co.za>
2010-04-09 01:53:44Yaroslav.Halchenkolinkissue7897 messages
2010-04-09 01:53:43Yaroslav.Halchenkocreate