Idle needs a unittest framework for module test modules. This means a test directory with at least one test module, a runtest module that successfully runs that test module, any new support functions needed for that test module, and inclusion of test_idle in Lib/test.

I am thinking of something like Lib/tkinter/test, but without subdirectories. I presume there should be something like tkinter/test/runtests, but I do not know if it reflects current best practice, nor whether it should be in idlelib or in idlelib/test.

One feature of runtests and the tkinter framework is separation of tests that do and do not require a gui. This requires cooperation of writers of each test module. Buildbots can only run tests that do not require a gui and nearly everyone else wants the same. On the other hand, we need automated gui tests too. A complete test of idlelib/runtest requires a test module with both kinds of tests.

List/test has test_tk. It runs tests with enable_gui=false unless run directly (and intentionally) as the main module. (Testing test_idle would also require tests in both modes.) It checks that tkinter is available. Should test_idle also check that idle is available? (On Windows, one can install both or neither. I don't know the situation on other systems.)

This issue was inspired by #12510 and is interdependent with it. As part of that issue, I would like to convert the expanded test set to run under unittest. #12510 needs a place to put a test_calltips module. On the other hand, this issue needs a test_x module to test runtests and test_idle.

tkinter/test/support has functions that can be used in idle tests, in particular, simulate_mouse_click(). I believe a needed addition is simulate_text_entry. That is needed to really finish the calltips tests. Currently, complete testing of tooltips requires non-automated hand tests.

Would simulate_mouse_click include selecting menu items, or would that be a separate function?

With such a test framework established, we could start adding tests as part of other issues, as is normal practice.

(Note: tkinter/test is pretty skeletal, and probably could use expansion, but except for support functions immediately needed for test_calltips, that is another issue.)