Message243188
This is valid:
℘· = 1
print(℘·)
#>>> 1
But this gives an error token:
from io import BytesIO
from tokenize import tokenize
stream = BytesIO("℘·".encode("utf-8"))
print(*tokenize(stream.read), sep="\n")
#>>> TokenInfo(type=56 (ENCODING), string='utf-8', start=(0, 0), end=(0, 0), line='')
#>>> TokenInfo(type=53 (ERRORTOKEN), string='℘', start=(1, 0), end=(1, 1), line='℘·')
#>>> TokenInfo(type=53 (ERRORTOKEN), string='·', start=(1, 1), end=(1, 2), line='℘·')
#>>> TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0), line='')
This is a continuation of http://bugs.python.org/issue9712. I'm not able to reopen the issue, so I thought I should report it anew.
It is tokenize that is wrong - Other_ID_Start and Other_ID_Continue are documented to be valid:
https://docs.python.org/3.5/reference/lexical_analysis.html#identifiers |
|
Date |
User |
Action |
Args |
2015-05-14 13:00:27 | Joshua.Landau | set | recipients:
+ Joshua.Landau |
2015-05-14 13:00:27 | Joshua.Landau | set | messageid: <1431608427.75.0.966529056278.issue24194@psf.upfronthosting.co.za> |
2015-05-14 13:00:27 | Joshua.Landau | link | issue24194 messages |
2015-05-14 13:00:27 | Joshua.Landau | create | |
|