This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Drekin
Recipients Drekin, benjamin.peterson, brett.cannon, eric.araujo, georg.brandl, gvanrossum, ncoghlan, pitrou, steve.dower, tshepang, vstinner
Date 2014-08-30.15:30:24
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1409412625.13.0.0204800496746.issue17620@psf.upfronthosting.co.za>
In-reply-to
Content
Antoine Pitrou: I understand. It would be nice to have that new Python string based readline hook. Its default implementation could be to call PyOS_Readline and decode the bytes using sys.stdin.encoding (as the tokenizer currently does). Tokenizer then woudn't need to decode if it called the new hook.

Victor Stinner: I'm going to try the approach of reencoding my stream to UTF-8. So then my UTF-16-LE encoded stream is decoded, then encoded to UTF-8, interpreted as null-terminated *char, which is returned to the tokenizer, which again decodes it and encodes to UTF-8. I wonder if the last step could be short-circuited. What is this UTF8 flag to Python parser? I couldn't find any information.
History
Date User Action Args
2014-08-30 15:30:25Drekinsetrecipients: + Drekin, gvanrossum, brett.cannon, georg.brandl, ncoghlan, pitrou, vstinner, benjamin.peterson, eric.araujo, tshepang, steve.dower
2014-08-30 15:30:25Drekinsetmessageid: <1409412625.13.0.0204800496746.issue17620@psf.upfronthosting.co.za>
2014-08-30 15:30:25Drekinlinkissue17620 messages
2014-08-30 15:30:24Drekincreate