Message366197
In my understanding of the C code that's what the C tokenizer is doing as well.
Here's the relevant snippet of the tokenizer (https://github.com/python/cpython/blob/4b222c9491d1700e9bdd98e6889b8d0ea1c7321e/Parser/tokenizer.c#L1358): When the tokenizer sees a valid identifier start, it goes into a loop that checks for a valid combination of string prefixes. If the combination is valid and it sees a quote directly after that, it goto's to the STRING-handling code. If not, then it breaks out of the loop and returns a NAME node.
Am I missing something? |
|
Date |
User |
Action |
Args |
2020-04-11 10:35:41 | lys.nikolaou | set | recipients:
+ lys.nikolaou, gvanrossum, serhiy.storchaka, pablogsal |
2020-04-11 10:35:41 | lys.nikolaou | set | messageid: <1586601341.05.0.137897249537.issue40246@roundup.psfhosted.org> |
2020-04-11 10:35:41 | lys.nikolaou | link | issue40246 messages |
2020-04-11 10:35:40 | lys.nikolaou | create | |
|