Message293789
Another example from Tools/i18n/pygettext.py, TokenEater::
def __call__(self, ttype, tstring, stup, etup, line):
# dispatch
## import token
## print >> sys.stderr, 'ttype:', token.tok_name[ttype], \
## 'tstring:', tstring
self.__state(ttype, tstring, stup[0])
...
eater.set_filename(filename)
try:
tokens = tokenize.tokenize(fp.readline)
for _token in tokens:
eater(*_token)
Another example using token.tok_name with token types coming from tokenize. |
|
Date |
User |
Action |
Args |
2017-05-16 21:09:26 | vstinner | set | recipients:
+ vstinner, georg.brandl, meador.inge, martin.panter, serhiy.storchaka, josephgordon |
2017-05-16 21:09:26 | vstinner | set | messageid: <1494968966.72.0.705937345545.issue25324@psf.upfronthosting.co.za> |
2017-05-16 21:09:26 | vstinner | link | issue25324 messages |
2017-05-16 21:09:26 | vstinner | create | |
|