When run at part of test suite on 3.5.0 (win7)
[341/397/4] test_tools
Usage: 2to3 [options] file|dir ...

regrtest.py: error: no such option: --slaveargs
test test_tools failed -- Traceback (most recent call last):
  File "C:\Programs\Python 3.5\lib\optparse.py", line 1386, in parse_args
    stop = self._process_args(largs, rargs, values)
  File "C:\Programs\Python 3.5\lib\optparse.py", line 1426, in _process_args
    self._process_long_opt(rargs, values)
  File "C:\Programs\Python 3.5\lib\optparse.py", line 1479, in _process_long_opt
    opt = self._match_long_opt(opt)
  File "C:\Programs\Python 3.5\lib\optparse.py", line 1464, in _match_long_opt
    return _match_abbrev(opt, self._long_opt)
  File "C:\Programs\Python 3.5\lib\optparse.py", line 1669, in _match_abbrev
    raise BadOptionError(s)
optparse.BadOptionError: no such option: --slaveargs

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Programs\Python 3.5\lib\test\test_tools\test_sundry.py", line 36, in test_sundry
    import_tool(name)
  File "C:\Programs\Python 3.5\lib\test\test_tools\__init__.py", line 22, in import_tool
    return importlib.import_module(toolname)
  File "C:\Programs\Python 3.5\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 662, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "C:\Programs\Python 3.5\Tools\scripts\2to3.py", line 5, in <module>
    sys.exit(main("lib2to3.fixes"))
  File "C:\Programs\Python 3.5\lib\lib2to3\main.py", line 179, in main
    options, args = parser.parse_args(args)
  File "C:\Programs\Python 3.5\lib\optparse.py", line 1388, in parse_args
    self.error(str(err))
  File "C:\Programs\Python 3.5\lib\optparse.py", line 1568, in error
    self.exit(2, "%s: error: %s\n" % (self.get_prog_name(), msg))
  File "C:\Programs\Python 3.5\lib\optparse.py", line 1558, in exit
    sys.exit(status)
SystemExit: 2

run by itselt (-m test test_tools)
PS C:\Users\Terry> py -3 -m test  test_tools
[1/1] test_tools
RefactoringTool: Skipping optional fixer: buffer
RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: set_literal
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: Can't open test_tools: [Errno 2] No such file or directory: 'test_tools'
RefactoringTool: No files need to be modified.
RefactoringTool: There was 1 error:
RefactoringTool: Can't open test_tools: [Errno 2] No such file or directory: 'test_tools'
Warning -- logging._handlerList was modified by test_tools
test test_tools failed -- Traceback (most recent call last):
  File "C:\Programs\Python 3.5\lib\test\test_tools\test_sundry.py", line 36, in test_sundry
    import_tool(name)
  File "C:\Programs\Python 3.5\lib\test\test_tools\__init__.py", line 22, in import_tool
    return importlib.import_module(toolname)
  File "C:\Programs\Python 3.5\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 662, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "C:\Programs\Python 3.5\Tools\scripts\2to3.py", line 5, in <module>
    sys.exit(main("lib2to3.fixes"))
SystemExit: 1

With 3.4.3, shorter message about multiple errors

Interesting, I got a different failure on 3.5.0:

======================================================================
ERROR: test_sundry (test.test_tools.test_sundry.TestSundryScripts)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Program Files\Python 3.5\lib\optparse.py", line 1386, in parse_args
    stop = self._process_args(largs, rargs, values)
  File "C:\Program Files\Python 3.5\lib\optparse.py", line 1430, in _process_args
    self._process_short_opts(rargs, values)
  File "C:\Program Files\Python 3.5\lib\optparse.py", line 1512, in _process_short_opts
    raise BadOptionError(opt)
optparse.BadOptionError: no such option: -u

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python 3.5\lib\test\test_tools\test_sundry.py", line 36, in test_sundry
    import_tool(name)
  File "C:\Program Files\Python 3.5\lib\test\test_tools\__init__.py", line 22, in import_tool
    return importlib.import_module(toolname)
  File "C:\Program Files\Python 3.5\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 662, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "C:\Program Files\Python 3.5\Tools\scripts\2to3.py", line 5, in <module>
    sys.exit(main("lib2to3.fixes"))
  File "C:\Program Files\Python 3.5\lib\lib2to3\main.py", line 179, in main
    options, args = parser.parse_args(args)
  File "C:\Program Files\Python 3.5\lib\optparse.py", line 1388, in parse_args
    self.error(str(err))
  File "C:\Program Files\Python 3.5\lib\optparse.py", line 1568, in error
    self.exit(2, "%s: error: %s\n" % (self.get_prog_name(), msg))
  File "C:\Program Files\Python 3.5\lib\optparse.py", line 1558, in exit
    sys.exit(status)
SystemExit: 2


I had no failures on 3.4.3.

> I had no failures on 3.4.3.

How do you run the test suite? The test fails when you pass option like -j0: python -m test -j0 test_tools.

IMHO the fix is trivial: monkey-patch sys.argv using support.swap_attr(sys, 'argv', ...). (I didn't try the fix).

For me now, test site fails on installed 3.6.6 and 3.7.0 and passes on repository builds.  Serial or parallel, no difference.

When I ran as part of the suite, I got --slaveargs message.

py -3.7 -m test  test_tools

says cannot import test tools.

py -3.7 -m test.test_tools
says the following for each file in the directory
Testing C:\Programs\Python37\Lib\test\<file>
and ends with
ERROR: test_sundry (test.test_tools.test_sundry.TestSundryScripts)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Programs\Python37\lib\test\test_tools\test_sundry.py", line 36, in test_sundry
    import_tool(name)
  File "C:\Programs\Python37\lib\test\test_tools\__init__.py", line 21, in import_tool
    return importlib.import_module(toolname)
  File "C:\Programs\Python37\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "C:\Programs\Python37\Tools\scripts\2to3.py", line 5, in <module>
    sys.exit(main("lib2to3.fixes"))
SystemExit: 2

Approving the behaviour on 3.6.5 and fresh cpython-built @master:

From the docs: test_sundry "contains extremely basic regression tests... which don't have separate tests of their own".

IMHO 2to3 can be added to the whitelist of test_sundry as it comes with its own unit tests. When test_sundry runs import_tool with '2to3', lib2to3 is called without args, resulting in an exit 2 which breaks the basic test.

Oh. I identified the root issue: Tools/scripts/2to3 is installed as Tools/scripts/2to3.py, a ".py" suffix is added. This script is tested on all non-Windows and skipped on Windows when run from source code, but test_sundry() imports 2to3.py which causes the test to fail, since the script expects an argument on the command line and fails with SystemExit which is not catched by test_sundry().

I wrote PR 8406 to skip 2to3.py. My PR also removes imported modules to remove side effects of the test.

--

Python 2.7 is not affected: test_tools is skipped on installed Python:

vstinner@WIN C:\>py -2.7 -m test test_tools                                     
...
test_tools skipped -- test irrelevant for an installed Python
...
Tests result: SUCCESS

New changeset 752d4b7531093c55d6f0a5846748f981d79b29d3 by Victor Stinner in branch 'master':
bpo-25094: Fix test_tools.test_sundry() on Windows (GH-8406)
https://github.com/python/cpython/commit/752d4b7531093c55d6f0a5846748f981d79b29d3

New changeset 1724c0c984e6406d80d8ebe3bb7b70f23c8b5f45 by Miss Islington (bot) in branch '3.6':
bpo-25094: Fix test_tools.test_sundry() on Windows (GH-8406)
https://github.com/python/cpython/commit/1724c0c984e6406d80d8ebe3bb7b70f23c8b5f45

New changeset ba1810e1ec8973e48128e17845b981d8894c0550 by Victor Stinner (Miss Islington (bot)) in branch '3.7':
bpo-25094: Fix test_tools.test_sundry() on Windows (GH-8406) (GH-8407)
https://github.com/python/cpython/commit/ba1810e1ec8973e48128e17845b981d8894c0550

The issue should now be fixed in 3.6, 3.7 and master branches. You will have to wait for the next release in each branch to get the fix! In the meanwhile, you can skip manually the 2to3 by modifying manually the test.

Thanks for everybody who helped to report and identify the origin of the bug!