When implementing an error handler, it must return a tuple consisting of a substitution string and a position where to resume decoding. In the case of the UTF-8 codec, the resume position is ignored, and it always resumes immediately after the character that caused the error.

To reproduce, use this code:

import codecs
codecs.register_error('err', lambda err: (b'x', err.end + 1))
assert repr(u'\uDD00yz'.encode('utf8', errors='err')) == b'xz'

The above code fails the assertion because the result is b'xyz'.

It works OK for some other codecs. I have not tried to make an exhaustive list of which ones work and which ones don't, therefore this problem might apply to others.

I forgot the quotes in the assertion, it should have been "b'xz'".

It works to me (after fixing the assertion).

What Python version do you use? In Python 2 u'\uDD00' is encodable to UTF-8, so the error handler is not called. u'\uDD00yz'.encode('utf8') gives '\xed\xb4\x80yz'.

Python 3.5 from Debian stretch (oldstable). You're right, I can't reproduce it in 3.7 from Buster. Sorry for the bogus report.