classification
Title: Silently skipped test in test_winsound
Type: behavior Stage: resolved
Components: Tests, Windows Versions: Python 3.4, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zach.ware Nosy List: brian.curtin, ezio.melotti, python-dev, r.david.murray, serhiy.storchaka, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2013-11-14 21:00 by zach.ware, last changed 2013-11-30 03:17 by zach.ware. This issue is now closed.

Files
File name Uploaded Description Edit
issue19595.diff zach.ware, 2013-11-19 18:02 review
Messages (8)
msg202887 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-11-14 21:00
In test_winsound, PlaySoundTest.test_alias_fallback has been commented out for quite some time but has been reported as 'ok' ever since.  Does anyone know what the current status of the test is?  Should it still be skipped (explicitly)?  Should it be marked as an expected failure?  Would it be dangerous to enable?  My own testing shows it to be harmless on my machine, but my range of test configurations is sorely limited at present.  My own suggestion is to mark it as an expected failure (or just enable it wholesale, and mark it if it does prove flaky) and see what the buildbots make of it, but I don't want to do that if it could cause anything worse than an exception.

Thoughts?
msg202897 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-11-14 21:44
I believe that expected failure will give an error report if the test succeeds.
msg202904 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-11-14 22:01
It doesn't appear to; at least on 3.4 an unexpected success is silently ignored by regrtest in non-verbose mode and added to the report in verbose mode (e.g., "OK (skipped=1, unexpected successes=1)") or when run alone.
msg203088 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-11-16 21:40
Maybe someone else can try it, or just enable it and see if the buildbots are happy (I think you can use custom builds as well for this kind of experiments).

> at least on 3.4 an unexpected success is silently ignored by regrtest
 
is this expected?
msg203405 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-11-19 18:02
I don't know that it's expected, but it doesn't appear to be unexpected.  regrtest doesn't appear to know anything about expected failures or unexpected successes, mostly because both concepts appear at the individual test level while regrtest deals at the test module level.  unittest docs don't actually say what happens when an expected failure fails to fail.

As for this issue, it looks like the skipping of the test predates _have_soundcard and has_sound.  If there are no objections, I'll try the attached patch on default; if it causes failures, I'll try a 'skipUnless(has_sound('SystemDefault')...', and if that work's I'll backport it to the other two branches.
msg204641 - (view) Author: Roundup Robot (python-dev) Date: 2013-11-28 05:58
New changeset 6f1c5d0b44ed by Zachary Ware in branch 'default':
Issue #19595: Re-enable a long-disabled test in test_winsound
http://hg.python.org/cpython/rev/6f1c5d0b44ed
msg204767 - (view) Author: Roundup Robot (python-dev) Date: 2013-11-30 03:15
New changeset 30b3798782f1 by Zachary Ware in branch '2.7':
Issue #19595: Re-enable a long-disabled test in test_winsound
http://hg.python.org/cpython/rev/30b3798782f1

New changeset 63f3e8670fa6 by Zachary Ware in branch '3.3':
Issue #19595: Re-enable a long-disabled test in test_winsound
http://hg.python.org/cpython/rev/63f3e8670fa6
msg204768 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-11-30 03:17
As long as the buildbots stay happy, this test is back to actually testing.
History
Date User Action Args
2013-11-30 03:17:03zach.waresetstatus: open -> closed
resolution: fixed
messages: + msg204768

stage: patch review -> resolved
2013-11-30 03:15:22python-devsetmessages: + msg204767
2013-11-28 05:58:44python-devsetnosy: + python-dev
messages: + msg204641
2013-11-19 18:02:34zach.waresetfiles: + issue19595.diff
messages: + msg203405

assignee: zach.ware
keywords: + patch
stage: test needed -> patch review
2013-11-16 21:40:52ezio.melottisetnosy: + ezio.melotti
messages: + msg203088
2013-11-14 22:01:58zach.waresetmessages: + msg202904
2013-11-14 21:44:27r.david.murraysetnosy: + r.david.murray
messages: + msg202897
2013-11-14 21:12:48zach.waresetstage: test needed
2013-11-14 21:00:56zach.warecreate