Author Sworddragon
Recipients Jan Niklas Hasse, Sworddragon, abarry, akira, barry, ezio.melotti, inada.naoki, lemburg, ncoghlan, r.david.murray, vstinner, yan12125
Date 2017-01-07.22:20:06
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1483827607.11.0.00354428831813.issue28180@psf.upfronthosting.co.za>
In-reply-to
Content
> $ cat badfilename.py 
> badfn = "こんにちは".encode('euc-jp').decode('utf-8', 'surrogateescape')
> print("bad filename:", badfn)
>
> $ PYTHONIOENCODING=utf-8:backslashreplace python3 badfilename.py 
> bad filename: \udca4\udcb3\udca4\udcf3\udca4ˤ\udcc1\udca4\udccf
>
> $ PYTHONIOENCODING=utf-8:surrogateescape python3 badfilename.py 
> bad filename: �����ˤ���

The first example is still readable (but effectively for an user not so much) while the second example appears to be not readable anymore at all. But the second example is actually technically still readable and there is no data loss, isn't it? As in this case it would probably not speak against surrogateescape for sys.stderr in UTF-8 non-strict mode. Otherwise backslashescape might be indeed the better choice.


I have thought about this a bit more and in case we go PEP 538 with keeping strict errors more or less the old way there might be another solution that could improve the overall issue: print() could get an option to allow changing the error handler on demand (with 'strict' still being the default).

Most things that I do output with print() are deterministic or optional and not important application data. Being able to print this information without caring for de-/encoding errors would mitigate this issue. In case application data is being printed where data loss is not desired exceptions can still be thrown.
History
Date User Action Args
2017-01-07 22:20:07Sworddragonsetrecipients: + Sworddragon, lemburg, barry, ncoghlan, vstinner, ezio.melotti, r.david.murray, inada.naoki, akira, yan12125, abarry, Jan Niklas Hasse
2017-01-07 22:20:07Sworddragonsetmessageid: <1483827607.11.0.00354428831813.issue28180@psf.upfronthosting.co.za>
2017-01-07 22:20:07Sworddragonlinkissue28180 messages
2017-01-07 22:20:06Sworddragoncreate