Author serhiy.storchaka
Recipients Arfrever, benjamin.peterson, brett.cannon, eric.snow, loewis, serhiy.storchaka
Date 2013-09-07.15:35:48
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1378568149.13.0.824239204214.issue18961@psf.upfronthosting.co.za>
In-reply-to
Content
Here is a file which is accepted by Python interpreter but is rejected by the tokenize module.

$ ./python nonutf8_coding_line.py 
$ ./python -m tokenize nonutf8_coding_line.py 
nonutf8_coding_line.py: error: invalid or missing encoding declaration for 'nonutf8_coding_line.py'

Python itself checks that a line is UTF-8 encoded only if not found a magic comment. The tokenize module checks it before searching (issue14629).
History
Date User Action Args
2013-09-07 15:35:49serhiy.storchakasetrecipients: + serhiy.storchaka, loewis, brett.cannon, benjamin.peterson, Arfrever, eric.snow
2013-09-07 15:35:49serhiy.storchakasetmessageid: <1378568149.13.0.824239204214.issue18961@psf.upfronthosting.co.za>
2013-09-07 15:35:49serhiy.storchakalinkissue18961 messages
2013-09-07 15:35:48serhiy.storchakacreate