Author michael.foord
Recipients Yaroslav.Halchenko, abingham, brian.curtin, exarkun, fperez, michael.foord, nchauvat, ncoghlan, pitrou, r.david.murray
Date 2011-07-20.14:16:57
SpamBayes Score 3.46057e-13
Marked as misclassified No
Message-id <1311171422.13.0.300016360764.issue7897@psf.upfronthosting.co.za>
In-reply-to
Content
Test selection would require load time parameterisation - although the current test selection mechanism is through importing which would probably *not* work without a specific fix. Same for run time parameterisation.

Well how *exactly* you generate the names is an open question, and once you've solved that problem it should be no harder to show them clearly in the failure message with a "single test report" than with multiple test reports.

The way to generate the names is to number each test *and* show the parameterised data as part of the name (which can lead to *huge* names if you're not careful - or just object reprs in names which isn't necessarily useful). I have a decorator example that does runtime parameterisation, concatenating failures to a single report but still keeping the generated name for each failure.

Another issue is whether or not parameterised tests share a TestCase instance or have separate ones. If you're doing load time generation it makes sense to have a separate test case instance, with setUp and tearDown run individually. This needs to be clearly documented as the parameter generation would run against an uninitialised (not setUp) testcase.

Obviously reporting multiple test failures separately (run time parameterisation) is a bit nicer, but runtime test generation doesn't play well with anything that works with test suites - where you expect all tests to be represented by a single test case instance in the suite. I'm not sure that's a solveable problem.
History
Date User Action Args
2011-07-20 14:17:02michael.foordsetrecipients: + michael.foord, exarkun, ncoghlan, pitrou, r.david.murray, brian.curtin, fperez, Yaroslav.Halchenko, nchauvat, abingham
2011-07-20 14:17:02michael.foordsetmessageid: <1311171422.13.0.300016360764.issue7897@psf.upfronthosting.co.za>
2011-07-20 14:16:58michael.foordlinkissue7897 messages
2011-07-20 14:16:58michael.foordcreate