I thought through my design criteria a bit, and in the process, decided on a specific concrete proposal to test.

1. It should be possible to run one test. It should easy to run (or not) the test (or tests) for a module when editing its file. It should be easy to tell when editing that there is a human test. These would both be served the a couple of lines in the 'if name' block (after the unittest lines) such as

    from idlelib.idle_test.htest import run
    run(GetCfgSectionNameDialog)  # or other class

2. It should be possible to see which modules have a human test. But I do not want a directory of 20+ 10-line files.

3. I want to factor out as much boilerplate code as possible. This will both make it easier to add tests and to modify the behavior of all tests at once.

Something like the following may work for both criteria.

In idle_test/htest.py

GetCfgSectionNameDialog_spec = {
  'kwds': {'title':'Get Name',
         'message':'Enter something',
         'used-names': {'abc'}},
  'msg': "After the text entered with [Ok] is stripped, <nothing>, "
         "'abc', or more that 30 chars are errors. "
         "Close with a valid entry (printed), [Cancel], or [X]"}

def run(klas):
  root = tk.Tk()
  klas_spec = globals[klas.__name__+'_arg']
  klas_spec.kwds['parent'] = root # does Idle use 'parent' consistently?
  def run_klas():
    widget = klas(**klas_spec.kwds)
    try:
      print(widget.result)
    except AttributeError:
      pass
  Message(root, text=klas_spec.msg).pack()
  Button(root, text='Test ' + klas.__name__, command=run_klas).pack()
  root.mainloop()

4. It should be possible to discover and run all tests. With one object per widget in htest, each with a marked name, it is easy to filter globals for marked objects.

  filter(lambda ob: hasattr(ob, '__name__') and '_args' in ob.__names__, globals())