This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author koffie
Recipients John Helour, koffie, lemburg, loewis, mdk, serhiy.storchaka, vstinner, xiang.zhang
Date 2021-06-24.17:18:21
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1624555101.86.0.586715133269.issue24339@roundup.psfhosted.org>
In-reply-to
Content
Hi Marc-Andre Lemburg,

Thanks for your reply. I tried using gencodec.py as could be downloaded from https://github.com/python/cpython/blob/main/Tools/unicode/gencodec.py as you mentioned. However the code in gencodec.py seems to be in a much worse shape than the iso6937.py attached here. The code in gencodec relies on being able to compare integers with tuples. This is caused by the lines:

mappings = sorted(map)

hinting that this code has never been run using python 3.

providing a decent sort key solves this issue. But after that other issues pop up. For example there seems to be some problems handling the 0x-001  by the not appropriately handling of items in the mapping that have MISSING_CODE resulting in things like:

    0x80: 0x-001

showing up in the generated code.

And then there is the issue that python_mapdef_code has as a side effect that it does 'del map["IDENTITY"]' causing "'IDENTITY' in map" in python_tabledef_code to always evaluate to False even when it should evaluate to True.

The problems above can be observed by just running gencodec.py on https://unicode.org/Public/MAPPINGS/VENDORS/APPLE/SYMBOL.TXT .

If gencodec.py was a trustworthy and well maintained piece of code, I would happily use it. However at the moment I don't see it as a valid option since debugging gencodec.py would cost me at least as much time as just writing its output myself instead of generating it. Additionally https://unicode.org/ doesn't seem to provide a mapping file for iso6937.

I do agree that using codecs.charmap_encode and codecs.charmap_decode is a much better solution then the one in iso6937.py. But I don't understand gencodec.py well enough to actually fix it.
History
Date User Action Args
2021-06-24 17:18:21koffiesetrecipients: + koffie, lemburg, loewis, vstinner, serhiy.storchaka, xiang.zhang, John Helour, mdk
2021-06-24 17:18:21koffiesetmessageid: <1624555101.86.0.586715133269.issue24339@roundup.psfhosted.org>
2021-06-24 17:18:21koffielinkissue24339 messages
2021-06-24 17:18:21koffiecreate