Having idle_test under idlelib is fine with me if it makes your life easier.

> 3. Many modules have more or useless 'sanity-check' tests run
> with the 'if __name__' mechanism. I think *all* of these should
> be replaced with unittest.main(xxx, verbosity=2, exit=False)
> to run the full test module. If the code module test has
> anything valuable that is not in the test module,
> it should be moved there; the rest should be deleted.

All the sanity checks (at least the reasonably sane) should be converted to unittest and moved from the modules to the respective test_* modules.  The others should be removed, and I don't think we have to keep the "if __name__" in the modules (the test modules should have it though).
Specifying the verbosity=2 in all the test modules might not be a good idea.  If you are running all the tests at once, that level of verbosity will just get in the way (and I'm not sure you can change it easily).  If you are running individual tests modules you should be able to change the verbosity from the command line.

> Question: is there a way for a test to detect being run by a buildbot?

You can handle this by adding a new resource to test.support.  The tests will normally be skipped unless you explicitly run them using "./python -m test -uhumanneeded" or you run the test file directly (you can enable the resource in the "if __name__").

> 4. I already used @template.txt to alter CallTips.py and start
> test_calltips.py and I want to use it for all the other 60 (or
> whatever) test files. I will (re)read the test-writing section
> and possibly suggest a patch.

As I suggested earlier, I think the "main" function in Lib/idlelib/CallTips.py should be converted to unittest and moved to test_calltips.  CallTips.py should not include any test-related code and no "if __name__" IMHO.

> But Idle-specific info and code does not belong there.

Agreed.  Actually once the initial conversion is done I don't think documenting it so important, since it will be easier to look at all the others tests and see what they do than finding the documentation that explain how things should be done.


> (5.) unittest.SkipTest is what I was looking for. I do not believe
> that support.import_module() will allow skipping if, or unless,
> the test is running on a specific platform. I need to look at #16935.

Nope, but you can do "if sys.platform() == '...': raise unittest.SkipTest(...)".  The TL;DR version of #16935 is that it will break test discovery on 3.3, but that's not a big problem.

> (6.) My concern and possible confusion about raising
> exceptions came from Jayakrishnan's patch

Either if you get a failure or an error, it still means there's something wrong :)