Author vstinner
Recipients ezio.melotti, vstinner
Date 2020-09-23.12:56:57
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1600865818.68.0.279745610542.issue41842@roundup.psfhosted.org>
In-reply-to
Content
Writing an unit test on the Python codecs machinery is facing a practical problem: there is no C nor Python API to unregister a codec search function.

It's even documented in a note of the codecs.register() function:

"Note: Search function registration is not currently reversible, which may cause problems in some cases, such as unit testing or module reloading."

https://docs.python.org/dev/library/codecs.html#codecs.register

test_codecs contains a long comment about that:

        # There's no way to unregister a codec search function, so we just
        # ensure we render this one fairly harmless after the test
        # case finishes by using the test case repr as the codec name
        # The codecs module normalizes codec names, although this doesn't
        # appear to be formally documented...
        # We also make sure we use a truly unique id for the custom codec
        # to avoid issues with the codec cache when running these tests
        # multiple times (e.g. when hunting for refleaks)

See bpo-22166 which fixed memory leaks in test_codecs.

In 2011, a Python user requested the function
https://mail.python.org/pipermail/python-dev/2011-September/113588.html

Marc-Andre Lemburg explained:

"There is no API to unregister a codec search function, since deregistration
would break the codec cache used by the registry to speedup codec
lookup."

One simple solution would be to clear the cache (PyInterpreterState.codec_search_cache) when codecs.unregister() removes a search function. I expect that calling unregister() is an uncommon operation, so the performance is not a blocker issue.
History
Date User Action Args
2020-09-23 12:56:58vstinnersetrecipients: + vstinner, ezio.melotti
2020-09-23 12:56:58vstinnersetmessageid: <1600865818.68.0.279745610542.issue41842@roundup.psfhosted.org>
2020-09-23 12:56:58vstinnerlinkissue41842 messages
2020-09-23 12:56:57vstinnercreate