Title: Add test to automatically detect missing format units in skipitem()
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.3
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: larry Nosy List: benjamin.peterson, larry, python-dev
Priority: normal Keywords: patch

Created on 2012-05-10 08:42 by larry, last changed 2012-06-22 19:59 by python-dev. This issue is now closed.

File name Uploaded Description Edit
larry.test_skipitem_parity.1.diff larry, 2012-05-10 08:42 review
larry.test_skipitem_parity.2.diff larry, 2012-06-06 00:03 review
larry.test_skipitem_parity.3.diff larry, 2012-06-20 06:58 Third patch review
larry.test_skipitem_parity.4.diff larry, 2012-06-21 09:08 Round four: fight! review
Messages (9)
msg160329 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2012-05-10 08:42
I recently fixed an old bug: some time ago someone added support for a new "format unit", 'Z', to PyArg_Parse(), but neglected to add matching support for it to skipitem().  Benjamin asked for a regression test.  Initially I said, okay, how the hell do I test that?  A quick grep showed that there was already a test like that, for the format unit 'C'.  I figured adding another one-off test was dumb, and there had to be a better way.

Attached is a patch against trunk that adds a new function to _testcapimodule.c: test_skipitem_parity().  The comment at the top describes says it best:

 * This function brute-force tests all** ASCII characters (1 to 127
 * inclusive) as format units, checking to see that
 * PyArg_ParseTupleAndKeywords() return consistent errors both when
 * the unit is attempted to be used and when it is skipped.  If the
 * format unit doesn't exist, we'll get one of two specific error
 * messages (one for used, one for skipped); if it does exist we
 * *won't* get that error--we'll get either no error or some other
 * error.  If we get the "does not exist" error for one test and
 * not for the other, there's a mismatch, and the test fails.
 *   ** Okay, it actually skips some ASCII characters.  Some characters
 *      have special funny semantics, and it would be difficult to
 *      accomodate them here.

I also removed the old test just for 'C', as this test subsumes that one.

Right now, the test runs to completion without complaint.  To test that it's really working, comment out an existing case statement in skipitem().  Or, add a new case statement, for a character that isn't otherwise supported (yet), to the top paragraph of case statements.  Doing either of these will cause a failure.

... you happy now, Benjamin?
msg160365 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-05-10 19:44
Can you see if you can write that test in Python? Perhaps simply providing a wrapper to cal PyArg_Parse with the arguments you want.
msg162382 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2012-06-06 00:03
I guess the answer was "no".

Attached is a new diff as you suggest.  It isn't really any shorter--in fact it's about 20% longer.  Though I freely admit it is more written-in-Python-y.

The test currently passes.  If you add support for a new format unit (like add "case '8':" above "case 'H':" in convertsimple()) without adding the corresponding case to simpleitem(), it fails.  If you perform the same operation on simpleitem() but not to convertsimple(), it again fails.  So I assert the test is working 100% correctly.
msg163003 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2012-06-17 01:05
Ping.  Benjamin, should I find another reviewer?
msg163260 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2012-06-20 06:58
Changes incorporated from Benjamin's feedback.  Thanks, Benjamin!

Is this ready to go?
msg163314 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2012-06-21 09:08
Incorporates Benjamin's self-admitted pedantic changes ;-)

Will it survive?  Will it get checked in?  Tune in... soon, I hope!
msg163356 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-06-21 18:17
msg163394 - (view) Author: Roundup Robot (python-dev) Date: 2012-06-22 10:57
New changeset ace45d23628a by Larry Hastings in branch 'default':
Issue #14769: test_capi now has SkipitemTest, which cleverly checks
msg163472 - (view) Author: Roundup Robot (python-dev) Date: 2012-06-22 19:59
New changeset 40d7869501a2 by Larry Hastings in branch 'default':
Issue #14769: Incorporated mildly pedantic feedback from python-dev.
Date User Action Args
2012-06-22 19:59:15python-devsetmessages: + msg163472
2012-06-22 10:58:00larrysetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2012-06-22 10:57:13python-devsetnosy: + python-dev
messages: + msg163394
2012-06-21 18:17:48benjamin.petersonsetmessages: + msg163356
2012-06-21 09:08:08larrysetfiles: + larry.test_skipitem_parity.4.diff

messages: + msg163314
2012-06-20 06:58:11larrysetfiles: + larry.test_skipitem_parity.3.diff

messages: + msg163260
2012-06-17 01:05:05larrysetmessages: + msg163003
2012-06-06 00:03:47larrysetfiles: + larry.test_skipitem_parity.2.diff

messages: + msg162382
2012-05-10 19:44:35benjamin.petersonsetmessages: + msg160365
2012-05-10 08:42:36larrycreate