This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author terry.reedy
Recipients benjamin.peterson, daniel.urban, eric.snow, ezio.melotti, gdr@garethrees.org, r.david.murray, terry.reedy, vladris
Date 2011-08-05.19:53:33
SpamBayes Score 0.01514403
Marked as misclassified No
Message-id <1312574014.54.0.075482283313.issue12675@psf.upfronthosting.co.za>
In-reply-to
Content
I have not used tokenize, but if it is *not* intended to exactly reproduce the internal tokenizer  behavior, the claim that it is should be amended.
History
Date User Action Args
2011-08-05 19:53:34terry.reedysetrecipients: + terry.reedy, benjamin.peterson, ezio.melotti, r.david.murray, daniel.urban, eric.snow, vladris, gdr@garethrees.org
2011-08-05 19:53:34terry.reedysetmessageid: <1312574014.54.0.075482283313.issue12675@psf.upfronthosting.co.za>
2011-08-05 19:53:34terry.reedylinkissue12675 messages
2011-08-05 19:53:33terry.reedycreate