This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Suspect test.test_codeccallbacks.test_mutatingdecodehandler
Type: enhancement Stage: needs patch
Components: Unicode Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, doerwalter, ezio.melotti, vstinner
Priority: normal Keywords:

Created on 2012-11-29 12:51 by amaury.forgeotdarc, last changed 2022-04-11 14:57 by admin.

Messages (2)
msg176645 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2012-11-29 12:51
The test created by 504084c04ac0 passes for bad reasons, at least for the second part:
- "test.mutating" error handler is never used.
- If I replace the last line of the test, it still passes, but the TypeError is raised by exc.object[:] = b"" (bytes are not mutable!)

I suggest to revert this old change completely. The premises ("the decoder might use memory that's no longer in use") are no longer true.
msg176661 - (view) Author: Walter Dörwald (doerwalter) * (Python committer) Date: 2012-11-29 16:48
True, the second test uses the wrong error handler.

And yes, you're correct, bytes are now immutable. And even if I try to decode a bytearray, what the callback gets to see is still an immutable bytes object::

   import codecs

   def mutating(exc):
      if isinstance(exc, UnicodeDecodeError):
         exc.object[:] = b""
         return ("\u4242", 0)
      else:
         raise TypeError("don't know how to handle %r" % exc)

   codecs.register_error('mutating', mutating)

   input = bytearray(b'bbb\xff')

   input.decode('ascii', 'mutating')

This still raises:

   TypeError: 'bytes' object does not support item assignment

So the change should indeed be reverted.
History
Date User Action Args
2022-04-11 14:57:38adminsetgithub: 60781
2016-04-21 03:15:44berker.peksagsetstage: needs patch
type: enhancement
versions: + Python 3.6
2012-11-29 16:48:11doerwaltersetnosy: + doerwalter
messages: + msg176661
2012-11-29 12:51:48amaury.forgeotdarccreate