This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author ncoghlan
Recipients akuchling, docs@python, gpolo, ncoghlan, terry.reedy
Date 2011-12-15.05:48:31
SpamBayes Score 6.7880395e-09
Marked as misclassified No
Message-id <1323928112.43.0.498634176463.issue2134@psf.upfronthosting.co.za>
In-reply-to
Content
Ah, I didn't read it as suggested documentation at all - you moved seamlessly from personal commentary to a docs suggestion without separating the two, so it appeared to be a complete non sequitur to me.

As for the docs suggestion, I think it works as the explanation of which tokens are affected once the concept of the token stream simplification is introduced:
=====
To simplify token stream handling, all literal tokens (':', '{', etc) are returned using the generic 'OP' token type. This allows them to be simply handled using common code paths (e.g. for literal transcription directly from input to output). Specific tokens can be distinguished by checking the "string" attribute of OP tokens for a match with the expected character sequence.

The affected tokens are all symbols and symbol combinations that are syntactically significant in expressions (as listed in the token module). Anything which is not an independent token (i.e. '#' which marks comments, '_' which is just part of a name, '\' which is used for line continuations, the contents of string literals and any symbols which are not a defined part of Python's syntax) is completely unaffected by this difference in behaviour.
===========

If "exact_type" is introduced in 3.3, then the first paragraph can be adjusted accordingly.
History
Date User Action Args
2011-12-15 05:48:32ncoghlansetrecipients: + ncoghlan, akuchling, terry.reedy, gpolo, docs@python
2011-12-15 05:48:32ncoghlansetmessageid: <1323928112.43.0.498634176463.issue2134@psf.upfronthosting.co.za>
2011-12-15 05:48:31ncoghlanlinkissue2134 messages
2011-12-15 05:48:31ncoghlancreate