Message301707
(Sorry, I didn't mean to challenge the authority of a core developer. I simply didn't notice that adding a comment reopens the issue. I hope this time I selected correct parameters and this doesn't happen again)
I'm trying to rephrase my concern.
Initially I thought there was a mistake in the tokenizer or in the token module.
After you pointed out the documentation about token.OP and exact_type, I'm worried about a smaller detail. The documentation only talks about *operators* and *delimiters* having type attribute set to token.OP. According to my understanding (and also the listings at https://docs.python.org/3/reference/lexical_analysis.html#operators), ellipsis is neither operator nor delimiter.
Am I right about this?
(I understand that the source representation of both ELLIPSIS and DOT tokens contains period *character(s)*, but I don't see why is this relevant when we discuss the properties of *tokens*)
Anyway, if ellipsis is neither operator nor delimiter, (and if there is a reason why it is treated similarly with operators and delimiters) then I recommend to update the documentation by replacing
> all Operators and Delimiters tokens
> are returned using the generic token.OP token type
with
> all Operators, Delimiters and Ellipsis tokens
> are returned using the generic token.OP token type
(I understand, that this is not a serious issue. If you prefer not to discuss it further then I'm happy to leave it as it is.) |
|
Date |
User |
Action |
Args |
2017-09-08 17:36:16 | Aivar.Annamaa | set | recipients:
+ Aivar.Annamaa, berker.peksag |
2017-09-08 17:36:16 | Aivar.Annamaa | set | messageid: <1504892176.6.0.37874711925.issue31394@psf.upfronthosting.co.za> |
2017-09-08 17:36:16 | Aivar.Annamaa | link | issue31394 messages |
2017-09-08 17:36:16 | Aivar.Annamaa | create | |
|