If you run test_idle using the standard Python regression test runner, regrtest, and use regrtest's -w option (to re-run failure test verbosely) *without* using regrtest's -j option (to run tests in separate processes), any real test failure triggering a rerun will cause a number of additional false IDLE test case failures because some test cases modify global state in such a way as to be not re-runable.

An example: at the moment there is a real IDLE test case failure (see Issue27830).  If I run test_idle with both -w and -j options:

$ /usr/local/bin/python3.6 -m test -w -j2 -uall test_idle
Run tests in parallel using 2 child processes
0:00:01 [1/1/1] test_idle failed
can't invoke "event" command:  application has been destroyed
    while executing
"event generate $w <<ThemeChanged>>"
    (procedure "ttk::ThemeChanged" line 6)
    invoked from within
"ttk::ThemeChanged"
test test_idle failed -- Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py", line 389, in test_click_help_source
    Equal(dialog.result, ('__test__', __file__))
AssertionError: Tuples differ: ('__test__', 'file:///Library/Frameworks/Python.framewo[58 chars].py') != ('__test__', '/Library/Frameworks/Python.framework/Vers[51 chars].py')

First differing element 1:
'file:///Library/Frameworks/Python.framewo[57 chars]y.py'
'/Library/Frameworks/Python.framework/Vers[50 chars]y.py'

  ('__test__',
-  'file:///Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')
?   -------

+  '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')
1 test failed:
    test_idle
Re-running failed tests in verbose mode
Re-running test 'test_idle' in verbose mode
test_autocomplete_event (idlelib.idle_test.test_autocomplete.AutoCompleteTest) ... ok
test_delayed_open_completions (idlelib.idle_test.test_autocomplete.AutoCompleteTest) ... ok
[...]
test_shell_show (idlelib.idle_test.test_warning.ShellWarnTest) ... ok
test_showwarnings (idlelib.idle_test.test_warning.ShellWarnTest) ... ok

======================================================================
FAIL: test_click_help_source (idlelib.idle_test.test_query.HelpsourceGuiTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py", line 389, in test_click_help_source
    Equal(dialog.result, ('__test__', __file__))
AssertionError: Tuples differ: ('__test__', 'file:///Library/Frameworks/Python.framewo[58 chars].py') != ('__test__', '/Library/Frameworks/Python.framework/Vers[51 chars].py')

First differing element 1:
'file:///Library/Frameworks/Python.framewo[57 chars]y.py'
'/Library/Frameworks/Python.framework/Vers[50 chars]y.py'

  ('__test__',
-  'file:///Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')
?   -------

+  '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')

----------------------------------------------------------------------
Ran 219 tests in 0.977s

FAILED (failures=1)

only the 1 valid test case failure shows up.

But if I run test_idle with just -w (no -j) options:

$ /usr/local/bin/python3.6 -m test -w -uall test_idle
Run tests sequentially
0:00:00 [1/1] test_idle
can't invoke "event" command:  application has been destroyed
    while executing
"event generate $w <<ThemeChanged>>"
    (procedure "ttk::ThemeChanged" line 6)
    invoked from within
"ttk::ThemeChanged"
test test_idle failed -- Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py", line 389, in test_click_help_source
    Equal(dialog.result, ('__test__', __file__))
AssertionError: Tuples differ: ('__test__', 'file:///Library/Frameworks/Python.framewo[58 chars].py') != ('__test__', '/Library/Frameworks/Python.framework/Vers[51 chars].py')

First differing element 1:
'file:///Library/Frameworks/Python.framewo[57 chars]y.py'
'/Library/Frameworks/Python.framework/Vers[50 chars]y.py'

  ('__test__',
-  'file:///Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')
?   -------

+  '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')

test_idle failed
1 test failed:
    test_idle
Re-running failed tests in verbose mode
Re-running test 'test_idle' in verbose mode
test_autocomplete_event (idlelib.idle_test.test_autocomplete.AutoCompleteTest) ... ok
test_delayed_open_completions (idlelib.idle_test.test_autocomplete.AutoCompleteTest) ... ok
[...]
test_no_delete (idlelib.idle_test.test_text.TkTextTest) ... ok
test_init_modal (idlelib.idle_test.test_textview.TextViewTest) ... ERROR
test_init_nonmodal (idlelib.idle_test.test_textview.TextViewTest) ... ERROR
test_ok (idlelib.idle_test.test_textview.TextViewTest) ... ERROR
test_view_file (idlelib.idle_test.test_textview.ViewFunctionTest) ... ok
test_view_text (idlelib.idle_test.test_textview.ViewFunctionTest) ... ok
ERROR
test_init (idlelib.idle_test.test_tree.TreeTest) ... ok
[...]
test_showwarnings (idlelib.idle_test.test_warning.ShellWarnTest) ... ok

======================================================================
ERROR: test_init_modal (idlelib.idle_test.test_textview.TextViewTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_textview.py", line 40, in setUp
    TV.transient.__init__()
NameError: name 'TV' is not defined

======================================================================
ERROR: test_init_nonmodal (idlelib.idle_test.test_textview.TextViewTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_textview.py", line 40, in setUp
    TV.transient.__init__()
NameError: name 'TV' is not defined

======================================================================
ERROR: test_ok (idlelib.idle_test.test_textview.TextViewTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_textview.py", line 40, in setUp
    TV.transient.__init__()
NameError: name 'TV' is not defined

======================================================================
ERROR: tearDownModule (idlelib.idle_test.test_textview)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_textview.py", line 26, in tearDownModule
    del TV
NameError: name 'TV' is not defined

======================================================================
FAIL: test_click_help_source (idlelib.idle_test.test_query.HelpsourceGuiTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py", line 389, in test_click_help_source
    Equal(dialog.result, ('__test__', __file__))
AssertionError: Tuples differ: ('__test__', 'file:///Library/Frameworks/Python.framewo[58 chars].py') != ('__test__', '/Library/Frameworks/Python.framework/Vers[51 chars].py')

First differing element 1:
'file:///Library/Frameworks/Python.framewo[57 chars]y.py'
'/Library/Frameworks/Python.framework/Vers[50 chars]y.py'

  ('__test__',
-  'file:///Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')
?   -------

+  '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_query.py')

----------------------------------------------------------------------
Ran 219 tests in 0.998s

FAILED (failures=1, errors=4)

now there are 4 or 5 errors in addition to the 1 failure.

The fifth false error is somewhat timing dependent, e.g. it shows up some of the time while the other 4 always show up:

======================================================================
ERROR: setUpClass (idlelib.idle_test.test_autocomplete.AutoCompleteTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/idle_test/test_autocomplete.py", line 34, in setUpClass
    macosx.setupApp(cls.root, None)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/macosx.py", line 245, in setupApp
    overrideRootMenu(root, flist)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/idlelib/macosx.py", line 215, in overrideRootMenu
    del mainmenu.menudefs[-1][1][0]
IndexError: list assignment index out of range


It's not a critical problem but it can be very confusing if you have not run into the problem before and can cause lost time trying to track down the false errors.