Message356610
When implementing an error handler, it must return a tuple consisting of a substitution string and a position where to resume decoding. In the case of the UTF-8 codec, the resume position is ignored, and it always resumes immediately after the character that caused the error.
To reproduce, use this code:
import codecs
codecs.register_error('err', lambda err: (b'x', err.end + 1))
assert repr(u'\uDD00yz'.encode('utf8', errors='err')) == b'xz'
The above code fails the assertion because the result is b'xyz'.
It works OK for some other codecs. I have not tried to make an exhaustive list of which ones work and which ones don't, therefore this problem might apply to others. |
|
Date |
User |
Action |
Args |
2019-11-14 14:30:25 | pgimeno | set | recipients:
+ pgimeno |
2019-11-14 14:30:24 | pgimeno | set | messageid: <1573741824.98.0.130776319775.issue38800@roundup.psfhosted.org> |
2019-11-14 14:30:24 | pgimeno | link | issue38800 messages |
2019-11-14 14:30:24 | pgimeno | create | |
|