Message189258
Attached is a 3.3 patch that I *believe* is ready to commit and push.
With my Win7 repository build, I tested running one test from a file (after "if __name__ == '__main__':") or command line ('python_d -m idlelib.PathBrowser'); all idle tests with an interactive interpreter (console or idle shell, see @template.txt for input); all idle tests from the command line ('python_d -m test.test_idle'); and idle tests as part of the CPython suite (python_d -m test). I also tested versions of all but the two batch-mode tests in my 3.3.1 installation.
From what i know, I do not believe there should be much issue with the framework working on non-Windows systems. But I would not mind verification. I presume this patch will work as is with 3.4, but ditto. 2.7 may need one tweak noted below. (Testing with 2.7 is difficult for me at the moment.) Nick: yes to your 2.7 plan. PEP 343 changes things.
Once applied to all three branches I think this patch is enough to close this issue and 'get the ball rolling'. Mock, gui testing, and any other big problems should be separate issues.
The patch adds
Itest/__init__.py (merges doc example, part of previous __init__.py)
Itest/@template (for both test_x.py and startup from x.py)
Itest/test_calltips.py (with first 2 of many tests)
Itest/test_pathbrowser.py (revised, with 1 test, see note below)
test/test_idle.py (revised skeleton of previous __init__.py)
and revises
CallTips.py
PathBrowser.py
to run the corresponding tests when run as '__main__'.
Question: "Unittest supports skipping individual test methods and even whole classes of tests" How about skipping (possibly conditionally) an entire file -- especially test_idle, which has no classes, or and test_xxx with multiple classes?
For multiple reasons related to the fact that Idle and idlelib *are* special, as recognized by PEP 343, I want to keep the individual test files in an idlelib subdirectory. as with tkinter tests. (I changed the name from 'test', so that 'import test' will a always import Lib/test.)
* Once in idlelib, changing to Itest is easier than to ../test/test_idle. With 62 .py files in idlelib, we will be opening pairs of files a lot.
* Copying code and test files to another directory, such as an installation idlelib/, will be easier. I will be doing that to run with new features and test things in the installation environment.
* PEP343 makes most of idlelib/* a private arena. Test/* is a public tree mainly for the buildbots. Tests put there are supposed to pass. In brief, I personally feel much more comfortable mucking around in a private arena with a small public window that can selectively open and closed as needed.
* We need an Itest directory anyway for things that do not belong in test/*. @template is an example, and I have in mind a couple of non-buildbot test scripts. We may also want an idle-specific support module, as tkinter has.
* Once people install Python so it runs, some still have problems running Idle. They ask a class instructor or someone else; post here, python-list, stackoverflow, or elsewhere; or give up. Sometimes the problem is with tkinter, sometimes with idle, sometimes with their knowledge. A good diagnosis script should save support time and user frustration. Both tkinter and idle tests should be available for this. The Windows installer makes the 17 mb of test/* optional.
Other changes from the previous patch:
* Use unittest.main to run tests.
* Guard test_idle execution with tkinter import, as it is _tkinter, not idlelib that might not be built. I left the guard for idlelib since someone who deletes idlelib/* might forget to delete test/test_idle.
* Exceptions raised outside of self.assertXyz (and self.fail) calls are regarded as an error in the test code rather than a failure of the code tested (26.3.4. Organizing test code). So, there being no 'assertNotRaises' context manager, I added "try:...except ...: self.fail()" to test_pathbrowser.py so a failure will be reported as such and not as an error. If there is a better way to do this, let me know.
Since 3.x chains exceptions and prints the original traceback, no message argument is needed with self.fail to explain the failure. For 2.7, I believe one should be added.
Note: to empirically test that a test fails properly, it must be run with code that does not work properly. This is a problem with writing tests after the fact for code that works properly.
I checked all 62 idlelib/*.py files for a test to see what we have to work with: the answer is 'not much'. Less than half the files have a test. All but 2 that do bring up a window and require a human to key, click, and look according to a script that is not provided. (This is not to say that the existing code will not be helpful for some gui tests.) One of the 2 remaining text tests prints multiple pages (a config file) for a person to check. By coincidence, the only automated tests are the ones in CallTips.py, the first file I looked at, last summer. |
|
Date |
User |
Action |
Args |
2013-05-15 00:11:37 | terry.reedy | set | recipients:
+ terry.reedy, ncoghlan, ned.deily, ezio.melotti, roger.serwy, r.david.murray, Todd.Rovito, tshepang, francismb, JayKrish, Tomoki.Imai |
2013-05-15 00:11:36 | terry.reedy | set | messageid: <1368576696.89.0.970421345099.issue15392@psf.upfronthosting.co.za> |
2013-05-15 00:11:36 | terry.reedy | link | issue15392 messages |
2013-05-15 00:11:36 | terry.reedy | create | |
|