Author serhiy.storchaka
Recipients Albert-Jan Nijburg, benjamin.peterson, matrixise, meador.inge, r.david.murray, serhiy.storchaka, vstinner
Date 2017-05-30.13:02:08
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
PR 1860 makes following files be generated from

* Include/token.h
* Parser/token.c. New file containing the array of token names _PyParser_TokenNames, and functions PyToken_OneChar(), PyToken_TwoChars(), PyToken_ThreeChars(), moved from Parser/tokenizer.c.
* Doc/library/ New file containing the list of constants, it is included in Doc/library/token.rst.

New Makefile target regen-token regenerates these files.

The dict EXACT_TOKEN_TYPES that maps operator strings to token names now is automatically generated and moved from to Tokens COMMENT, NL and ENCODING used only in now are added in as in issue25324.
Date User Action Args
2017-05-30 13:02:08serhiy.storchakasetrecipients: + serhiy.storchaka, vstinner, benjamin.peterson, r.david.murray, meador.inge, matrixise, Albert-Jan Nijburg
2017-05-30 13:02:08serhiy.storchakasetmessageid: <>
2017-05-30 13:02:08serhiy.storchakalinkissue30455 messages
2017-05-30 13:02:08serhiy.storchakacreate