Message301688
Thank you for the report, but this behavior is already documented at https://docs.python.org/3/library/tokenize.html
To simplify token stream handling, all Operators and Delimiters
tokens are returned using the generic token.OP token type. The
exact type can be determined by checking the exact_type property
on the named tuple returned from tokenize.tokenize().
If you replace the following line
print(ellipsis)
with
print(ellipsis.exact_type)
you will see that it prints '52'. |
|
Date |
User |
Action |
Args |
2017-09-08 09:40:34 | berker.peksag | set | recipients:
+ berker.peksag, Aivar.Annamaa |
2017-09-08 09:40:34 | berker.peksag | set | messageid: <1504863634.52.0.764611409416.issue31394@psf.upfronthosting.co.za> |
2017-09-08 09:40:34 | berker.peksag | link | issue31394 messages |
2017-09-08 09:40:34 | berker.peksag | create | |
|