Attached is a 3.3 patch that I *believe* is ready to commit and push.
With my Win7 repository build, I tested running one test from a file (after "if __name__ == '__main__':") or command line ('python_d -m idlelib.PathBrowser'); all idle tests with an interactive interpreter (console or idle shell, see @template.txt for input); all idle tests from the command line ('python_d -m test.test_idle'); and idle tests as part of the CPython suite (python_d -m test). I also tested versions of all but the two batch-mode tests in my 3.3.1 installation.

From what i know, I do not believe there should be much issue with the framework working on non-Windows systems. But I would not mind verification. I presume this patch will work as is with 3.4, but ditto. 2.7 may need one tweak noted below. (Testing with 2.7 is difficult for me at the moment.) Nick: yes to your 2.7 plan. PEP 343 changes things.

Once applied to all three branches I think this patch is enough to close this issue and 'get the ball rolling'. Mock, gui testing, and any other big problems should be separate issues.

The patch adds
Itest/__init__.py  (merges doc example, part of previous __init__.py)
Itest/@template  (for both test_x.py and startup from x.py)
Itest/test_calltips.py  (with first 2 of many tests)
Itest/test_pathbrowser.py  (revised, with 1 test, see note below)
test/test_idle.py  (revised skeleton of previous __init__.py)

and revises 
CallTips.py
PathBrowser.py
to run the corresponding tests when run as '__main__'.

Question: "Unittest supports skipping individual test methods and even whole classes of tests" How about skipping (possibly conditionally) an entire file -- especially test_idle, which has no classes, or and test_xxx with multiple classes?

For multiple reasons related to the fact that Idle and idlelib *are* special, as recognized by PEP 343, I want to keep the individual test files in an idlelib subdirectory. as with tkinter tests. (I changed the name from 'test', so that 'import test' will a always import Lib/test.)

* Once in idlelib, changing to Itest is easier than to ../test/test_idle. With 62 .py files in idlelib, we will be opening pairs of files a lot.

* Copying code and test files to another directory, such as an installation idlelib/, will be easier. I will be doing that to run with new features and test things in the installation environment.

* PEP343 makes most of idlelib/* a private arena. Test/* is a public tree mainly for the buildbots. Tests put there are supposed to pass. In brief, I personally feel much more comfortable mucking around in a private arena with a small public window that can selectively open and closed as needed.

* We need an Itest directory anyway for things that do not belong in test/*. @template is an example, and I have in mind a couple of non-buildbot test scripts. We may also want an idle-specific support module, as tkinter has.

* Once people install Python so it runs, some still have problems running Idle. They ask a class instructor or someone else; post here, python-list, stackoverflow, or elsewhere; or give up. Sometimes the problem is with tkinter, sometimes with idle, sometimes with their knowledge. A good diagnosis script should save support time and user frustration. Both tkinter and idle tests should be available for this. The Windows installer makes the 17 mb of test/* optional.

Other changes from the previous patch:
* Use unittest.main to run tests.

* Guard test_idle execution with tkinter import, as it is _tkinter, not idlelib that might not be built. I left the guard for idlelib since someone who deletes idlelib/* might forget to delete test/test_idle.

* Exceptions raised outside of self.assertXyz (and self.fail) calls are regarded as an error in the test code rather than a failure of the code tested (26.3.4. Organizing test code). So, there being no 'assertNotRaises' context manager, I added "try:...except ...: self.fail()" to test_pathbrowser.py so a failure will be reported as such and not as an error. If there is a better way to do this, let me know.

Since 3.x chains exceptions and prints the original traceback, no message argument is needed with self.fail to explain the failure. For 2.7, I believe one should be added.

Note: to empirically test that a test fails properly, it must be run with code that does not work properly. This is a problem with writing tests after the fact for code that works properly.

I checked all 62 idlelib/*.py files for a test to see what we have to work with: the answer is 'not much'. Less than half the files have a test. All but 2 that do bring up a window and require a human to key, click, and look according to a script that is not provided. (This is not to say that the existing code will not be helpful for some gui tests.) One of the 2 remaining text tests prints multiple pages (a config file) for a person to check. By coincidence, the only automated tests are the ones in CallTips.py, the first file I looked at, last summer.