Message396505
Hi Marc-Andre Lemburg,
Thanks for your reply. I tried using gencodec.py as could be downloaded from https://github.com/python/cpython/blob/main/Tools/unicode/gencodec.py as you mentioned. However the code in gencodec.py seems to be in a much worse shape than the iso6937.py attached here. The code in gencodec relies on being able to compare integers with tuples. This is caused by the lines:
mappings = sorted(map)
hinting that this code has never been run using python 3.
providing a decent sort key solves this issue. But after that other issues pop up. For example there seems to be some problems handling the 0x-001 by the not appropriately handling of items in the mapping that have MISSING_CODE resulting in things like:
0x80: 0x-001
showing up in the generated code.
And then there is the issue that python_mapdef_code has as a side effect that it does 'del map["IDENTITY"]' causing "'IDENTITY' in map" in python_tabledef_code to always evaluate to False even when it should evaluate to True.
The problems above can be observed by just running gencodec.py on https://unicode.org/Public/MAPPINGS/VENDORS/APPLE/SYMBOL.TXT .
If gencodec.py was a trustworthy and well maintained piece of code, I would happily use it. However at the moment I don't see it as a valid option since debugging gencodec.py would cost me at least as much time as just writing its output myself instead of generating it. Additionally https://unicode.org/ doesn't seem to provide a mapping file for iso6937.
I do agree that using codecs.charmap_encode and codecs.charmap_decode is a much better solution then the one in iso6937.py. But I don't understand gencodec.py well enough to actually fix it. |
|
Date |
User |
Action |
Args |
2021-06-24 17:18:21 | koffie | set | recipients:
+ koffie, lemburg, loewis, vstinner, serhiy.storchaka, xiang.zhang, John Helour, mdk |
2021-06-24 17:18:21 | koffie | set | messageid: <1624555101.86.0.586715133269.issue24339@roundup.psfhosted.org> |
2021-06-24 17:18:21 | koffie | link | issue24339 messages |
2021-06-24 17:18:21 | koffie | create | |
|