Report more silently skipped tests as skipped #63771

zware · 2013-11-13T17:18:58Z

BPO	19572
Nosy	@terryjreedy, @pitrou, @vstinner, @ned-deily, @ezio-melotti, @voidspace, @zware, @serhiy-storchaka, @vajrasky
Files	skiptest_not_return_or_pass.diff skiptest_not_return_or_pass.v2.diff: A couple additional skips skiptest_not_return_or_pass.v3.diff: Version 3 skiptest_not_return_or_pass.v4-3.3.diff skiptest_not_return_or_pass.v4-2.7.diff skiptest_not_return_or_pass.v5-2.7.diff: 2.7, version 5 skiptest_not_return_or_pass.v6-2.7.diff: Version 6, the real version 5!

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zware'
closed_at = <Date 2013-12-10.20:11:00.043>
created_at = <Date 2013-11-13.17:18:58.144>
labels = ['type-feature', 'tests']
title = 'Report more silently skipped tests as skipped'
updated_at = <Date 2013-12-12.16:33:38.122>
user = 'https://github.com/zware'

bugs.python.org fields:

activity = <Date 2013-12-12.16:33:38.122>
actor = 'python-dev'
assignee = 'zach.ware'
closed = True
closed_date = <Date 2013-12-10.20:11:00.043>
closer = 'zach.ware'
components = ['Tests']
creation = <Date 2013-11-13.17:18:58.144>
creator = 'zach.ware'
dependencies = []
files = ['32599', '32601', '32625', '32728', '32761', '33080', '33082']
hgrepos = []
issue_num = 19572
keywords = ['patch']
message_count = 23.0
messages = ['202762', '202769', '202774', '202793', '202795', '202906', '203076', '203080', '203334', '203494', '203526', '203647', '205522', '205523', '205533', '205836', '205838', '205839', '205844', '205849', '205851', '205863', '205959']
nosy_count = 10.0
nosy_names = ['terry.reedy', 'pitrou', 'vstinner', 'ned.deily', 'ezio.melotti', 'michael.foord', 'python-dev', 'zach.ware', 'serhiy.storchaka', 'vajrasky']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue19572'
versions = ['Python 2.7']

zware · 2013-11-13T17:18:51Z

(Nosy list copied from bpo-18702)

Grepping for "^\s+return$" and "^\s+pass$" in Lib/test turned up several more tests that are silently skipped (along with many legitimate uses of each). The attached patch turns each into a skip in a few various ways, including splitting a test into two tests in a couple of cases.

I'll make a few comments on Rietveld myself to point out places where I would really like some extra scrutiny.

serhiy-storchaka · 2013-11-13T19:05:59Z

See also bpo-19492 and bpo-19493.

zware · 2013-11-13T20:24:40Z

Starting a review of bpo-19492, I realized my original regexs didn't take comments into account. Grepping "^\s+return\s*(#.*)?$" instead turned up a couple more skips, including some in test_tempfile that said "return # ugh, can't use SkipTest", relics of the days before unittest.SkipTest.

serhiy-storchaka · 2013-11-13T22:35:18Z

I added some comments on Rietveld to the first patch. In general it LGTM. I will made a review for additional skips tomorrow.

I suggest extract importlib related changes in separated issue. Bratt Cannon should review them.

serhiy-storchaka · 2013-11-13T22:44:10Z

I will made a review for additional skips tomorrow.

Additional skips LGTM.

zware · 2013-11-14T22:05:50Z

Here's a new patch to address the reviews from Serhiy and Ezio (thanks to you both!).

ezio-melotti · 2013-11-16T18:59:28Z

I get a failure on Linux with test_posix:
$ ./python -m test test_posix
[1/1] test_posix
test test_posix failed -- Traceback (most recent call last):
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 713, in test_getcwd_long_pathnames
    _create_and_do_getcwd(dirname)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 708, in _create_and_do_getcwd
    _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
  File "/home/wolf/dev/py/py3k/Lib/test/test_posix.py", line 706, in _create_and_do_getcwd
    os.getcwd()
OSError: [Errno 34] Numerical result out of range

1 test failed:
test_posix

serhiy-storchaka · 2013-11-16T20:38:36Z

Except pickletester and test_posix third patch LGTM. test_posix is worth separate issue. test_reduce and test_getinitargs in pickletester are always empty and can be just removed (if you don't want implement them).

zware · 2013-11-19T03:20:30Z

I'll open new issues for test_posix and pickletester and commit the rest of the patch as soon as I can get it backported. Thanks for the reviews!

zware · 2013-11-20T15:42:19Z

pickletester issue opened at bpo-19648. The test_posix issue already has an open issue at bpo-9246.

serhiy-storchaka · 2013-11-20T19:48:51Z

skiptest_not_return_or_pass.v4-3.3.diff LGTM.

zware · 2013-11-21T15:53:51Z

The 2.7 patch has a lot of extra changes in it, extra review is probably in order for it.

python-dev · 2013-12-08T06:45:25Z

New changeset 3283fb24106d by Zachary Ware in branch '3.3':
bpo-19572: More silently skipped tests explicitly skipped.
http://hg.python.org/cpython/rev/3283fb24106d

New changeset 03afd2d7d395 by Zachary Ware in branch 'default':
bpo-19572: More silently skipped tests explicitly skipped.
http://hg.python.org/cpython/rev/03afd2d7d395

zware · 2013-12-08T06:46:38Z

Committed on 3.3 and default; I'd still like some extra eyes on the 2.7 patch before I commit it.

serhiy-storchaka · 2013-12-08T09:21:55Z

I have added few comments on Rietveld.

zware · 2013-12-10T17:25:09Z

Here's a new 2.7 patch. It addresses Serhiy's review comments and doesn't change test_xpickle, which I will be opening a new issue for.

zware · 2013-12-10T17:34:41Z

I missed the comments on test_bsddb; I'll either post a new patch here or open a new issue depending on how big that diff becomes.

zware · 2013-12-10T17:46:30Z

This patch includes Serhiy's suggestions. Oops!

serhiy-storchaka · 2013-12-10T18:55:49Z

$ ./python -3 -m test.regrtest test_builtin
test_builtin
test test_builtin crashed -- <type 'exceptions.AssertionError'>: filter ('.+ is renamed to imp.reload', DeprecationWarning) did not catch any warning
1 test failed:
    test_builtin

Rest of the skiptest_not_return_or_pass.v6-2.7.diff patch LGTM.

python-dev · 2013-12-10T20:09:41Z

New changeset 423e09aedf79 by Zachary Ware in branch '2.7':
Issue bpo-19572: More silently skipped tests explicitly skipped.
http://hg.python.org/cpython/rev/423e09aedf79

zware · 2013-12-10T20:11:00Z

Fixed that last comment in the commit. Thank you for all the reviews, Serhiy!

python-dev · 2013-12-10T22:07:00Z

New changeset ca9bca7aecda by Zachary Ware in branch '2.7':
Issue bpo-19572: Replace a return that shouldn't have been removed from test_os.
http://hg.python.org/cpython/rev/ca9bca7aecda

python-dev · 2013-12-12T16:33:38Z

New changeset 1ad2ff119356 by Zachary Ware in branch '3.3':
Avoid UnicodeEncodeError by only printing ASCII.
http://hg.python.org/cpython/rev/1ad2ff119356

zware added tests Tests in the Lib/test dir type-feature A feature request or enhancement labels Nov 13, 2013

zware self-assigned this Dec 8, 2013

zware closed this as completed Dec 10, 2013

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Report more silently skipped tests as skipped #63771

Report more silently skipped tests as skipped #63771

zware commented Nov 13, 2013

zware commented Nov 13, 2013

serhiy-storchaka commented Nov 13, 2013

zware commented Nov 13, 2013

serhiy-storchaka commented Nov 13, 2013

serhiy-storchaka commented Nov 13, 2013

zware commented Nov 14, 2013

ezio-melotti commented Nov 16, 2013

serhiy-storchaka commented Nov 16, 2013

zware commented Nov 19, 2013

zware commented Nov 20, 2013

serhiy-storchaka commented Nov 20, 2013

zware commented Nov 21, 2013

python-dev mannequin commented Dec 8, 2013

zware commented Dec 8, 2013

serhiy-storchaka commented Dec 8, 2013

zware commented Dec 10, 2013

zware commented Dec 10, 2013

zware commented Dec 10, 2013

serhiy-storchaka commented Dec 10, 2013

python-dev mannequin commented Dec 10, 2013

zware commented Dec 10, 2013

python-dev mannequin commented Dec 10, 2013

python-dev mannequin commented Dec 12, 2013

Report more silently skipped tests as skipped #63771

Report more silently skipped tests as skipped #63771

Comments

zware commented Nov 13, 2013

zware commented Nov 13, 2013

serhiy-storchaka commented Nov 13, 2013

zware commented Nov 13, 2013

serhiy-storchaka commented Nov 13, 2013

serhiy-storchaka commented Nov 13, 2013

zware commented Nov 14, 2013

ezio-melotti commented Nov 16, 2013

serhiy-storchaka commented Nov 16, 2013

zware commented Nov 19, 2013

zware commented Nov 20, 2013

serhiy-storchaka commented Nov 20, 2013

zware commented Nov 21, 2013

python-dev mannequin commented Dec 8, 2013

zware commented Dec 8, 2013

serhiy-storchaka commented Dec 8, 2013

zware commented Dec 10, 2013

zware commented Dec 10, 2013

zware commented Dec 10, 2013

serhiy-storchaka commented Dec 10, 2013

python-dev mannequin commented Dec 10, 2013

zware commented Dec 10, 2013

python-dev mannequin commented Dec 10, 2013

python-dev mannequin commented Dec 12, 2013