classification
Title: Add tests to verify API match of modules with 2 implementations
Type: enhancement Stage: resolved
Components: Tests Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: BreamoreBoy, belopolsky, berker.peksag, brett.cannon, eric.araujo, gregory.p.smith, laura, pitrou, python-dev, r.david.murray, stutzbach, terry.reedy
Priority: low Keywords: patch

Created on 2010-09-15 13:46 by stutzbach, last changed 2015-04-14 22:25 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
issue9859.patch laura, 2015-04-14 16:21 review
issue9859_1.patch laura, 2015-04-14 19:23 review
Messages (16)
msg116445 - (view) Author: Daniel Stutzbach (stutzbach) (Python committer) Date: 2010-09-15 13:46
Recently it came to light that the classes in C and Python implementations of the io module have slightly different attributes (issue9858).  I propose the addition of a helper function in Lib/test/support.py to verify that the classes in two different implementations define the same attributes.  Then, we can add tests to use that function to verify that C and Python implementations define the same API (for the io module, but also for other modules where we have two implementations).  The script I added to issue9858 could serve as a starting point for such a function.

Since CPython's standard library is the de facto reference implementation, it's important that it define one API and not two slightly different ones. :-)
msg116452 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-09-15 14:56
Shouldn't the test suite catch such discrepancies by testing all of the API?  So your script catching something would be the equivalent of "oops, we forgot a test" (or "oops, this name shouldn't be public").  Which is not a bad thing to have as part of the test suite.
msg116453 - (view) Author: Daniel Stutzbach (stutzbach) (Python committer) Date: 2010-09-15 15:03
Yes, exactly. :-)

(see also Issue9731, which has a similar flavor)
msg116752 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-09-18 03:32
+1
I presume you can use the experience with #9858 to either refine the program to have fewer false positives or refine the program doc to explain the possibility where such are unavoidable. (And as a lint-like metatest, I think false positives are better than false negatives.)
msg221823 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-28 23:57
@Daniel do you intend putting forward a formal patch on this issue?  I'm asking as I think issue9858 is effectively completed and I've just asked for a formal patch review on Issue9731.
msg240932 - (view) Author: Laura Rupprecht (laura) * Date: 2015-04-14 16:21
Created a patch to check whether classes define the same attributes.

This adds a test for RawIOBase, where Python and and C implementations are out of sync, but skips one of the tests as the issue (issue9858) is still outstanding.
msg240938 - (view) Author: Daniel Stutzbach (stutzbach) (Python committer) Date: 2015-04-14 16:35
With regret, I have not had time to work on patches and am unlikely to have time in the near future.
msg240941 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2015-04-14 16:46
Thanks for the patch! I've reviewed it on Rietveld: http://bugs.python.org/review/9859/
msg240993 - (view) Author: Laura Rupprecht (laura) * Date: 2015-04-14 19:23
Thanks Berker!

I've added some updates to the patch taking your suggestions into account. There is also a change from using "detect_module_matches" to "detect_api_mismatch", as it is a more general (and accurate) description.

Also, adding tests for the "detect_api_mismatch" helper function.
msg241002 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-04-14 19:58
New changeset b42d1f2aa7a2 by Gregory P. Smith in branch 'default':
issue9859: Adds a test.support.detect_api_mismatch function useful to
https://hg.python.org/cpython/rev/b42d1f2aa7a2

New changeset 0b6c894c3c96 by Gregory P. Smith in branch 'default':
issue9859: Adds a CPyMatchTest test case to compare the exposed APIs
https://hg.python.org/cpython/rev/0b6c894c3c96
msg241004 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2015-04-14 20:00
I've added a couple of comments about the test: http://bugs.python.org/review/9859/
msg241008 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2015-04-14 20:06
Thanks!  Patch applied.  I reworded one doc string slightly and fixed up a few lines that were longer than 80 characters.  Berker's most recent comments are good ones and can be addressed in another patch.

Laura, can you jump through the https://www.python.org/psf/contrib/contrib-form/ hoop?  (it'll add the "*" next to your name in the bug tracker showing the contributor form has been signed)

Still TODO: fix the actual issue of the APIs being different. (issue9858)
msg241012 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-04-14 20:26
New changeset 9903368b9d7b by Gregory P. Smith in branch 'default':
issue9859: rename CPyMatchTest to APIMismatchTest and add @support.cpython_only.
https://hg.python.org/cpython/rev/9903368b9d7b

New changeset cbdd56d07123 by Gregory P. Smith in branch 'default':
issue9859: Document test.support.detect_api_mismatch() and simplify its test.
https://hg.python.org/cpython/rev/cbdd56d07123
msg241026 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-04-14 20:54
New changeset c6df85e1d42e by Gregory P. Smith in branch 'default':
issue9859: Use an expected failure rather than a skip.
https://hg.python.org/cpython/rev/c6df85e1d42e
msg241036 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2015-04-14 21:45
Thanks :)
msg241046 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-04-14 22:25
New changeset 0316811f33b2 by Gregory P. Smith in branch 'default':
issue9859: add the missing versionadded tag to the documentation.
https://hg.python.org/cpython/rev/0316811f33b2
History
Date User Action Args
2015-04-14 22:25:19python-devsetmessages: + msg241046
2015-04-14 21:45:54berker.peksagsetmessages: + msg241036
stage: patch review -> resolved
2015-04-14 21:39:01gregory.p.smithsetstatus: open -> closed
resolution: fixed
2015-04-14 20:54:20python-devsetmessages: + msg241026
2015-04-14 20:26:47python-devsetmessages: + msg241012
2015-04-14 20:06:49gregory.p.smithsetmessages: + msg241008
2015-04-14 20:00:54berker.peksagsetmessages: + msg241004
2015-04-14 19:58:58python-devsetnosy: + python-dev
messages: + msg241002
2015-04-14 19:23:46laurasetfiles: + issue9859_1.patch

messages: + msg240993
2015-04-14 16:46:59berker.peksagsetversions: + Python 3.5, - Python 3.2
nosy: + berker.peksag

messages: + msg240941

stage: needs patch -> patch review
2015-04-14 16:40:03gregory.p.smithsetassignee: gregory.p.smith

nosy: + gregory.p.smith
2015-04-14 16:35:20stutzbachsetmessages: + msg240938
2015-04-14 16:21:10laurasetfiles: + issue9859.patch

nosy: + laura
messages: + msg240932

keywords: + patch
2014-06-28 23:57:22BreamoreBoysetnosy: + BreamoreBoy
messages: + msg221823
2011-05-07 15:29:55eric.araujosetnosy: + brett.cannon, belopolsky, pitrou, eric.araujo
2010-09-18 03:32:07terry.reedysetnosy: + terry.reedy
messages: + msg116752
2010-09-15 15:03:22stutzbachsetmessages: + msg116453
2010-09-15 14:56:06r.david.murraysetnosy: + r.david.murray
messages: + msg116452
2010-09-15 13:46:35stutzbachcreate