➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Drekin
Recipients Drekin, abarnert, ezio.melotti, martin.panter, vstinner
Date 2016-01-20.20:48:07
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1453322887.57.0.804314405345.issue26152@psf.upfronthosting.co.za>
In-reply-to
Content 
It could still point to the first or the last byte of the invalid token rather than to the start of the next token. Also, by the Python implementation of the tokenizer in tokenize module we get an ERRORTOKEN containing a non-breaking space followed by a number token containing 2.
History
Date User Action Args
2016-01-20 20:48:07Drekinsetrecipients: + Drekin, vstinner, ezio.melotti, martin.panter, abarnert
2016-01-20 20:48:07Drekinsetmessageid: <1453322887.57.0.804314405345.issue26152@psf.upfronthosting.co.za>
2016-01-20 20:48:07Drekinlinkissue26152 messages
2016-01-20 20:48:07Drekincreate