classification
Title: Report skipped ctypes tests as skipped
Type: enhancement Stage: resolved
Components: ctypes, Tests Versions: Python 3.3, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zach.ware Nosy List: amaury.forgeotdarc, belopolsky, meador.inge, python-dev, serhiy.storchaka, zach.ware
Priority: normal Keywords: patch

Created on 2013-11-04 12:57 by serhiy.storchaka, last changed 2014-07-23 19:40 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
skip_tests_ctypes.patch serhiy.storchaka, 2013-11-04 12:57 review
skip_tests_ctypes.v2.diff zach.ware, 2013-11-15 08:14 review
skip_tests_ctypes.v3.diff zach.ware, 2013-12-19 18:12 review
Messages (11)
msg202124 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-11-04 12:57
Some skipped ctypes tests are reported as passed. Proposed patch adds explicit reporting them as skipped.

See also issue18702.
msg202775 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-11-13 20:44
Grepping with the same regexes I used for #19572 come up with some extra skips in the ctypes tests too; would you like me to create a new patch including them or would you like to do it, Serhiy?
msg202791 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-11-13 22:30
> Grepping with the same regexes I used for #19572 come up with some extra
> skips in the ctypes tests too; would you like me to create a new patch
> including them or would you like to do it, Serhiy?

Please do this Zachary.
msg202926 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-11-15 08:14
Here's a patch.  It turned out to be much more extensive than I expected, and the diff turned into a huge huge ugly monster that I hope Rietveld can help to make sense of, since the majority of the diff is simply changes in indentation level.

The most major change of this patch is the addition of a "needs_symbol" decorator to ctypes.test.__init__, which is then used throughout the tests.  Pre-patch, the tests are littered with constructs like this:

"""
class TestSomething(unittest.TestCase):
    try:
        c_wchar
    except NameError:
        pass
    else:
        def test_something_using_c_wchar(self):
            ...
"""

These have all (I think) been simplified to:

"""
class TestSomething(unittest.TestCase):
    @needs_symbol('c_wchar')
    def test_something_using_c_wchar(self):
        ...
"""


There are also several instances of tests guarded by "if sys.platform = 'win32':" or similar, which have been turned into @unittest.skipUnless, and several tests which were commented out which have been uncommented and given unconditional skips.

On my Gentoo machine at home, the patch takes the ctypes test suite from 348 tests with 1 skip to 422 tests with 78 skips.
msg206628 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-12-19 18:26
Here's a new patch addressing your review comment, Serhiy.  It also addresses some failures on Windows in test_values: Win_ValuesTestCase depends on 'pydll' being defined in the module toplevel and shadowing ctypes.pydll; this definition was removed some years ago (before ctypes was merged into Python).  I replaced each instance of 'pydll' with 'pythonapi', which makes the tests pass (with appropriate update to test_frozentable's expectations), but I don't understand all of ctypes well enough to be sure that it is definitely the correct fix.

Also, a few long lines that were already touched have been split (without messing with other long lines) and a couple of tests have been converted from "def X_test" to "def test_X" with an unconditional skip.  Another empty file has also been removed: test_errcheck is completely commented out except for a couple of imports, so it is removed entirely.  The test that calls doctest.testmod in test_objects has been adjusted to fail the test if any of the doctests fail, and to not care about sys.version.  Finally, test_wintypes has been rearranged to skip the test class rather than the module on non-Windows platforms to keep the same number of tests between platforms.
msg206643 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-12-19 20:21
I can't say anything about pydll, other changes LGTM. Except that I'm not sure that test_wintypes needs a fix.
msg206649 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-12-19 21:34
I'd prefer to keep the change to test_wintypes, simply because I was rather surprised to find an extra test being run on Windows.

As for the pydll/pythonapi issue, any thoughts from Amaury, Meador, or Alexander?  The relevant change that removed the definition of pydll in test_values can be found here[1].  That also makes me wonder why exactly the test is named "*Win*_ValuesTestCase" and whether it actually needs a skip or just a rename, since there doesn't appear to me to be anything related to Windows about the test.


[1] http://ctypes.cvs.sourceforge.net/viewvc/ctypes/ctypes/ctypes/test/test_values.py?hideattic=0&r1=1.1.2.1&r2=1.1.2.2
msg220479 - (view) Author: Roundup Robot (python-dev) Date: 2014-06-13 18:48
New changeset 6f63fff5c120 by Zachary Ware in branch '3.4':
Issue #19493: Refactor ctypes test package.
http://hg.python.org/cpython/rev/6f63fff5c120

New changeset 86d14cf2a6a8 by Zachary Ware in branch 'default':
Issue #19493: Merge with 3.4
http://hg.python.org/cpython/rev/86d14cf2a6a8
msg220488 - (view) Author: Roundup Robot (python-dev) Date: 2014-06-13 19:40
New changeset 08a2b36f6287 by Zachary Ware in branch '2.7':
Issue #19493: Backport 6f63fff5c120
http://hg.python.org/cpython/rev/08a2b36f6287
msg220489 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-06-13 19:43
Committed; thanks for the review, Serhiy.
msg223769 - (view) Author: Roundup Robot (python-dev) Date: 2014-07-23 19:40
New changeset 49a2bed5185a by Zachary Ware in branch '2.7':
Issue #19493: Fix two uses of ctypes.test.requires (it's not a decorator)
http://hg.python.org/cpython/rev/49a2bed5185a

New changeset 374a9a259c09 by Zachary Ware in branch '3.4':
Issue #19493: Fix two uses of ctypes.test.requires (it's not a decorator)
http://hg.python.org/cpython/rev/374a9a259c09
History
Date User Action Args
2014-07-23 19:40:59python-devsetmessages: + msg223769
2014-06-13 19:43:25zach.waresetstatus: open -> closed
resolution: fixed
messages: + msg220489

stage: patch review -> resolved
2014-06-13 19:40:41python-devsetmessages: + msg220488
2014-06-13 18:48:36python-devsetnosy: + python-dev
messages: + msg220479
2013-12-19 21:34:11zach.waresetmessages: + msg206649
2013-12-19 20:21:44serhiy.storchakasetmessages: + msg206643
2013-12-19 18:26:05zach.waresetmessages: + msg206628
2013-12-19 18:12:43zach.waresetfiles: + skip_tests_ctypes.v3.diff
2013-12-14 23:14:01serhiy.storchakasetassignee: zach.ware
2013-11-15 08:14:53zach.waresetfiles: + skip_tests_ctypes.v2.diff

messages: + msg202926
2013-11-13 22:30:18serhiy.storchakasetmessages: + msg202791
2013-11-13 20:44:14zach.waresetnosy: + zach.ware
messages: + msg202775
2013-11-04 12:57:58serhiy.storchakacreate