classification
Title: test for token.py, and consistency tests for tokenize.py
Type: enhancement Stage: needs patch
Components: Tests Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Jim Fasarakis-Hilliard, ammar2, r.david.murray
Priority: normal Keywords: easy

Created on 2017-03-14 23:03 by r.david.murray, last changed 2017-03-16 03:17 by r.david.murray.

Pull Requests
URL Status Linked Edit
PR 681 open ammar2, 2017-03-16 02:19
Messages (3)
msg289628 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-03-14 23:03
http://bugs.python.org/issue24622 made reminded me that a while back we added tests for the keyword module that includes a test that if you run it, you get the result that is checked in.  The same thing could be done for the token.py module.  And then we could add a cross-check test that tokenize.py has all the symbols defined as well.
msg289702 - (view) Author: Ammar Askar (ammar2) * Date: 2017-03-16 02:19
This is my first real substantial testing change so I'd appreciate all feedback. The way I did the cross-check doesn't actually rely on any of the information from the test_keyword style regeneration test. 

I think this approach is a lot simpler, and will prevent mistakes like the one seen in http://bugs.python.org/issue24622
msg289704 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-03-16 03:17
The cross check test itself doesn't depend on a regeneration, but it does depend on the information in token.h.  Meanwhile the validity of *that* is checked by regeneration in your test_token tests.  This is exactly what I had in mind :).

So, I haven't done a full review, but the quick glance I took looked good.
History
Date User Action Args
2017-03-16 03:17:31r.david.murraysetmessages: + msg289704
2017-03-16 02:19:46ammar2setnosy: + ammar2
messages: + msg289702
2017-03-16 02:19:29ammar2setpull_requests: + pull_request558
2017-03-15 09:50:53Jim Fasarakis-Hilliardsetnosy: + Jim Fasarakis-Hilliard
2017-03-14 23:03:41r.david.murraycreate