Message226140
Antoine Pitrou: I understand. It would be nice to have that new Python string based readline hook. Its default implementation could be to call PyOS_Readline and decode the bytes using sys.stdin.encoding (as the tokenizer currently does). Tokenizer then woudn't need to decode if it called the new hook.
Victor Stinner: I'm going to try the approach of reencoding my stream to UTF-8. So then my UTF-16-LE encoded stream is decoded, then encoded to UTF-8, interpreted as null-terminated *char, which is returned to the tokenizer, which again decodes it and encodes to UTF-8. I wonder if the last step could be short-circuited. What is this UTF8 flag to Python parser? I couldn't find any information. |
|
Date |
User |
Action |
Args |
2014-08-30 15:30:25 | Drekin | set | recipients:
+ Drekin, gvanrossum, brett.cannon, georg.brandl, ncoghlan, pitrou, vstinner, benjamin.peterson, eric.araujo, tshepang, steve.dower |
2014-08-30 15:30:25 | Drekin | set | messageid: <1409412625.13.0.0204800496746.issue17620@psf.upfronthosting.co.za> |
2014-08-30 15:30:25 | Drekin | link | issue17620 messages |
2014-08-30 15:30:24 | Drekin | create | |
|