Message189264
A few comments about the patch:
1) I would prefer a more explicit idle_test instead of Itest (assuming "I" means idle);
2) having idle_test in Lib/test would be better if possible (see #10572);
3) the tests should only be in idle_test, and not in the "if __name__ == '__main__':" of the files;
4) I'm not sure the @template file is necessary. It could be documented somewhere else though (see e.g. http://docs.python.org/3/library/test.html#writing-unit-tests-for-the-test-package).
IOW putting all tests in a separate dir is a good thing, and the dir could either be in Lib/test or in Lib/idlelib. All the tests should be inside this dir, except for Lib/test/test_idle.py that should be the entry point used to run all the tests in idle_test (see e.g. the tests for ctypes, email, and json). Individual tests in idle_test can be run on their own, and they should use the "if __name__ == '__main__': unittest.main()" idiom.
> Question: "Unittest supports skipping individual test
> methods and even whole classes of tests" How about skipping
> (possibly conditionally) an entire file -- especially test_idle,
> which has no classes, or and test_xxx with multiple classes?
You can raise unittest.SkipTest or use support.import_module(). This works fine if you run the tests through regrtest, but be aware that unittest itself only sees this as a skip from 3.4 (see #16935).
> * Exceptions raised outside of self.assertXyz (and self.fail)
> calls are regarded as an error in the test code rather than a
> failure of the code tested (26.3.4. Organizing test code).
> So, there being no 'assertNotRaises' context manager, I added
> "try:...except ...: self.fail()" to test_pathbrowser.py so a
> failure will be reported as such and not as an error. If
> there is a better way to do this, let me know.
If it's supposed to work the try/except is not necessary IMHO. By this logic every line of code should be wrapped in a try/except :)
> Since 3.x chains exceptions and prints the original traceback,
> no message argument is needed with self.fail to explain the
> failure. For 2.7, I believe one should be added.
If you still want to keep the try/except, I would provide a meaningful failure message in any case.
> Note: to empirically test that a test fails properly, it must
> be run with code that does not work properly. This is a
> problem with writing tests after the fact for code that works
> properly.
It's not difficult to break the code to test that test works though :)
> I checked all 62 idlelib/*.py files for a test to see what we
> have to work with: the answer is 'not much'. Less than half
> the files have a test.
If possible I think these should all be moved to idle_test. You can also add an additional resource to regrtest to skip the ones that require manual intervention. |
|
Date |
User |
Action |
Args |
2013-05-15 03:05:30 | ezio.melotti | set | recipients:
+ ezio.melotti, terry.reedy, ncoghlan, ned.deily, roger.serwy, r.david.murray, Todd.Rovito, tshepang, francismb, JayKrish, Tomoki.Imai |
2013-05-15 03:05:30 | ezio.melotti | set | messageid: <1368587130.83.0.536853435031.issue15392@psf.upfronthosting.co.za> |
2013-05-15 03:05:30 | ezio.melotti | link | issue15392 messages |
2013-05-15 03:05:30 | ezio.melotti | create | |
|