➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Albert-Jan Nijburg
Recipients Albert-Jan Nijburg, georg.brandl, josephgordon, martin.panter, meador.inge, serhiy.storchaka, vstinner
Date 2017-05-24.11:16:28
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1495624588.26.0.361267539916.issue25324@psf.upfronthosting.co.za>
In-reply-to
Content 
I've updated the PR and added the tokenize tokens to token.h and their names to tokenizer.c. This way they'll show up when you run token.py. The names will always be in tok_name and tokenizer.py will use those. Not breaking the public api and no longer modifying token.py when you import tokenizer.py.
History
Date User Action Args
2017-05-24 11:16:28Albert-Jan Nijburgsetrecipients: + Albert-Jan Nijburg, georg.brandl, vstinner, meador.inge, martin.panter, serhiy.storchaka, josephgordon
2017-05-24 11:16:28Albert-Jan Nijburgsetmessageid: <1495624588.26.0.361267539916.issue25324@psf.upfronthosting.co.za>
2017-05-24 11:16:28Albert-Jan Nijburglinkissue25324 messages
2017-05-24 11:16:28Albert-Jan Nijburgcreate