➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Albert-Jan Nijburg
Recipients Albert-Jan Nijburg, georg.brandl, josephgordon, martin.panter, meador.inge, serhiy.storchaka, vstinner
Date 2017-05-16.21:24:31
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1494969871.78.0.448359439899.issue25324@psf.upfronthosting.co.za>
In-reply-to
Content 
> I prefer to add tokenize tokens directly in Lib/token.py, and then get COMMENT, NL and ENCODING using tok_name.index().

That would make more sense from a breaking change perspective, but we would step on the toes of anyone adding `COMMENT`, `NL`, or `ENCODING` to `token.h` because `token.py` is generated from that.

It would also make much more sense to have them as fields on `token` if they are in `tok_name` in `token`.
History
Date User Action Args
2017-05-16 21:24:31Albert-Jan Nijburgsetrecipients: + Albert-Jan Nijburg, georg.brandl, vstinner, meador.inge, martin.panter, serhiy.storchaka, josephgordon
2017-05-16 21:24:31Albert-Jan Nijburgsetmessageid: <1494969871.78.0.448359439899.issue25324@psf.upfronthosting.co.za>
2017-05-16 21:24:31Albert-Jan Nijburglinkissue25324 messages
2017-05-16 21:24:31Albert-Jan Nijburgcreate