This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author ezio.melotti
Recipients JayKrish, Todd.Rovito, Tomoki.Imai, ezio.melotti, francismb, ncoghlan, ned.deily, r.david.murray, roger.serwy, terry.reedy, tshepang
Date 2013-05-15.03:05:30
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1368587130.83.0.536853435031.issue15392@psf.upfronthosting.co.za>
In-reply-to
Content
A few comments about the patch:
1) I would prefer a more explicit idle_test instead of Itest (assuming "I" means idle);
2) having idle_test in Lib/test would be better if possible (see #10572);
3) the tests should only be in idle_test, and not in the "if __name__ == '__main__':" of the files;
4) I'm not sure the @template file is necessary.  It could be documented somewhere else though (see e.g. http://docs.python.org/3/library/test.html#writing-unit-tests-for-the-test-package).

IOW putting all tests in a separate dir is a good thing, and the dir could either be in Lib/test or in Lib/idlelib.  All the tests should be inside this dir, except for Lib/test/test_idle.py that should be the entry point used to run all the tests in idle_test (see e.g. the tests for ctypes, email, and json).  Individual tests in idle_test can be run on their own, and they should use the "if __name__ == '__main__': unittest.main()" idiom.

> Question: "Unittest supports skipping individual test 
> methods and even whole classes of tests" How about skipping
> (possibly conditionally) an entire file -- especially test_idle,
> which has no classes, or and test_xxx with multiple classes?

You can raise unittest.SkipTest or use support.import_module().  This works fine if you run the tests through regrtest, but be aware that unittest itself only sees this as a skip from 3.4 (see #16935).

> * Exceptions raised outside of self.assertXyz (and self.fail)
> calls are regarded as an error in the test code rather than a
> failure of the code tested (26.3.4. Organizing test code).
> So, there being no 'assertNotRaises' context manager, I added
> "try:...except ...: self.fail()" to test_pathbrowser.py so a
> failure will be reported as such and not as an error. If
> there is a better way to do this, let me know.

If it's supposed to work the try/except is not necessary IMHO.  By this logic every line of code should be wrapped in a try/except :)

> Since 3.x chains exceptions and prints the original traceback,
> no message argument is needed with self.fail to explain the
> failure. For 2.7, I believe one should be added.

If you still want to keep the try/except, I would provide a meaningful failure message in any case.

> Note: to empirically test that a test fails properly, it must
> be run with code that does not work properly. This is a
> problem with writing tests after the fact for code that works
> properly.

It's not difficult to break the code to test that test works though :)

> I checked all 62 idlelib/*.py files for a test to see what we
> have to work with: the answer is 'not much'. Less than half
> the files have a test.

If possible I think these should all be moved to idle_test.  You can also add an additional resource to regrtest to skip the ones that require manual intervention.
History
Date User Action Args
2013-05-15 03:05:30ezio.melottisetrecipients: + ezio.melotti, terry.reedy, ncoghlan, ned.deily, roger.serwy, r.david.murray, Todd.Rovito, tshepang, francismb, JayKrish, Tomoki.Imai
2013-05-15 03:05:30ezio.melottisetmessageid: <1368587130.83.0.536853435031.issue15392@psf.upfronthosting.co.za>
2013-05-15 03:05:30ezio.melottilinkissue15392 messages
2013-05-15 03:05:30ezio.melotticreate