New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Python does not support the GEORGIAN-PS charset #63658
Comments
LANG=ka_GE.georgianps /usr/bin/python3
Fatal Python error: Py_Initialize: Unable to get the locale encoding
LookupError: unknown encoding: GEORGIAN-PS
Aborted (core dumped) but with python-2.7.5 no crash...
(fedora 19) |
This bug was initially reported in LibreOffice: |
I found three georgian encodings: http://tools.ietf.org/html/draft-giasher-geostd8-00 The first one ("GEORGIAN-PS") is probably the most accurate because it is the one included in the GNU libc. Could you please try to copy attached georgian_ps.py file into /usr/lib64/python3.3/encodings/ (or /usr/lib/python3.3/encodings/ for 32-bit Linux)? Then try to print georgian letters using: print(bytes(range(0xc0, 0xe6)).decode("GEORGIAN-PS")) Please give me also your locale encoding: import locale; print(locale.getpreferredencoding()) @caolán: Do you know the GEORGIAN-ACADEMY encoding? It doesn't look to be used by any glibc locale. On my Fedora 18, I have 3 georgian locales:
You can workaround this issue by switching your locale from ka_GE.georgianps to ka_GE.utf8. |
With recent versions of Python (e.g. 3.9) this no longer causes a crash. Python apparently falls back to UTF-8, at least on my system: $ LANG=ka_GE.georgianps python3.9
Python 3.9.7 (default, Sep 9 2021, 23:20:13)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale; print(locale.getpreferredencoding())
UTF-8 I'm marking this as fixed. If someone still has issues with this encoding, please open a new issue with up-to-date information. |
Python uses UTF-8 if the locale is not supported: $ LANG=xxx python3.9 -c "import sys; print(sys.flags.utf8_mode)"
1 On Fedora 34, the locale is still supported, and Python 3.11 still fails: vstinner@apu$ LANG=ka_GE.georgianps locale vstinner@apu$ LANG=ka_GE.georgianps python3.11 -c "import sys; print(sys.flags.utf8_mode)" Current thread 0x00007ff89b81d2c0 (most recent call first): |
Possible solutions (they can be combined):
|
On 19.10.2021 10:44, Serhiy Storchaka wrote:
As with all encodings we add: there has to be a real need to support IMO, we should not really add more encodings to the stdlib, but instead https://pypi.org/project/python-iconv/ Perhaps we ought to make it easier for such packages to provide
I think this would be a more general solution to such cases, provided |
Without this unicode range spec every character is deferred through an external `iconv` call, slowing down the BuildModel code tremendously!
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: