This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author JustinTArthur
Recipients JustinTArthur
Date 2019-09-04.23:21:42
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1567639302.72.0.55214965787.issue38032@roundup.psfhosted.org>
In-reply-to
Content
Python 3 code with an identifier that has a non-spacing mark in it does not get tokenized by lib2to3 and will result in an exception thrown in the parsing process.

Parsing the attached file (badvar.py), results in `ParseError: bad token: type=58, value='̇', context=('', (1, 1))`

This happens because the Name pattern regular expression in lib2to3 is `r'\w+'` and the word character class doesn't contain non-spacing marks (and possible other [continuation characters allowed in Python 3 identifiers](https://docs.python.org/3/reference/lexical_analysis.html#identifiers)).

(reported by energizer in the Python IRC channel)
History
Date User Action Args
2019-09-04 23:21:42JustinTArthursetrecipients: + JustinTArthur
2019-09-04 23:21:42JustinTArthursetmessageid: <1567639302.72.0.55214965787.issue38032@roundup.psfhosted.org>
2019-09-04 23:21:42JustinTArthurlinkissue38032 messages
2019-09-04 23:21:42JustinTArthurcreate