classification
Title: LookupError: unknown encoding: X-MAC-JAPANESE
Type: Stage:
Components: Unicode Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: hyeshik.chang Nosy List: ezio.melotti, gvanrossum, hyeshik.chang, josm, lemburg, mokeefe, olivier-berten
Priority: normal Keywords: patch

Created on 2007-10-13 10:07 by josm, last changed 2012-06-27 12:12 by olivier-berten. This issue is now closed.

Files
File name Uploaded Description Edit
x_mac_japanese.diff josm, 2007-10-13 10:07
maccjkcodecs-1-trunk.diff hyeshik.chang, 2008-06-26 12:05 Macintosh CJK codecs for 2.6 trunk
maccjkcodecs-1-py3k.diff hyeshik.chang, 2008-06-27 05:42 Macintosh CJK codecs for 3.0
cjkmactemporary.diff hyeshik.chang, 2008-08-20 06:09 temporary patch that adds aliases to existing codecs
Messages (10)
msg56386 - (view) Author: John Smith (josm) Date: 2007-10-13 10:07
When I compile Python-3.0a1 on Mac OS X with Japanese locale,
I've got LookupError like below.

==========================================
running build_scripts
creating build/scripts-3.0
Traceback (most recent call last):
  File "./setup.py", line 1572, in <module>
    main()
  File "./setup.py", line 1567, in main
    'Lib/smtpd.py']
  File "/private/tmp/Python-3.0a1/Lib/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/private/tmp/Python-3.0a1/Lib/distutils/dist.py", line 943, in
run_commands
    self.run_command(cmd)
  File "/private/tmp/Python-3.0a1/Lib/distutils/dist.py", line 963, in
run_command
    cmd_obj.run()
  File "/private/tmp/Python-3.0a1/Lib/distutils/command/build.py", line
106, in run
    self.run_command(cmd_name)
  File "/private/tmp/Python-3.0a1/Lib/distutils/cmd.py", line 317, in
run_command
    self.distribution.run_command(command)
  File "/private/tmp/Python-3.0a1/Lib/distutils/dist.py", line 963, in
run_command
    cmd_obj.run()
  File
"/private/tmp/Python-3.0a1/Lib/distutils/command/build_scripts.py", line
51, in run
    self.copy_scripts()
  File
"/private/tmp/Python-3.0a1/Lib/distutils/command/build_scripts.py", line
82, in copy_scripts
    first_line = f.readline()
  File "/private/tmp/Python-3.0a1/Lib/io.py", line 1259, in readline
    decoder = self._decoder or self._get_decoder()
  File "/private/tmp/Python-3.0a1/Lib/io.py", line 1111, in _get_decoder
    make_decoder = codecs.getincrementaldecoder(self._encoding)
  File "/private/tmp/Python-3.0a1/Lib/codecs.py", line 951, in
getincrementaldecoder
    decoder = lookup(encoding).incrementaldecoder
LookupError: unknown encoding: X-MAC-JAPANESE
make: *** [sharedmods] Error 1
==========================================

This problem happens for lack of appropriate codec
so also occurs in apps using getdefaultencoding.

After patching Tools/unicode/Makefile and
running make generates build/mac_japanese.py, mac-japanese codec.
msg56449 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-10-15 17:27
Marc-Andre, do you understand this issue? Your name is in
Tools/unicode/Makefile; the patch deletes the line

$(RM) build/mac_japanese.*

from the apple target, which seems rather arbitrary.
msg56455 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2007-10-15 17:56
My name appears in that Makefile because I wrote it and used it to
create the charmap codecs.

The reason why the Mac Japanese codec was not created for 2.x was the
size of the mapping table. 

Ideal would be to have the C version of the CJK codecs support the Mac
Japanese encoding as well. Adding back the charmap based Mac Japanese
codec would be a compromise.

The absence of the Mac Japanese codec causes (obvious) problems for many
Japanese Python users running Mac OS X.
msg56456 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2007-10-15 17:57
Adding Python 2.6 as version target.
msg62974 - (view) Author: Hyeshik Chang (hyeshik.chang) * (Python committer) Date: 2008-02-25 06:14
I'll take this.
msg68780 - (view) Author: Hyeshik Chang (hyeshik.chang) * (Python committer) Date: 2008-06-26 12:05
Added a patch that implements codecs for CJK Macintosh encodings.
I tried to implement that just alike the other existing CJK codecs,
but it required many inefficient mapping tables due to their odd
mappings (like this: u'ABCDE' <-> 'ab' AND u'ABCD' <-> 'ac'!).

So, I decided to implement a general extension codec wrapper that
can be easily modified by dictionaries given by Python code.
Because all Mac CJK encodings have codecs that implement their base
encodings, I just put their difference in Python codec code.
The extension mechanism may be reused in customized codecs for
in-house applications or legacy encoding supports.

The first patch was generated for 2.6 trunk.  I'm working on porting
it to 3.0.
msg71802 - (view) Author: Hyeshik Chang (hyeshik.chang) * (Python committer) Date: 2008-08-23 08:08
Committed patch "cjkmactemporary.diff" as r65988 in the py3k branch.
I'll open another issue for cjkcodecs implementation of Mac codecs.
msg71940 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2008-08-25 15:32
On 2008-08-23 10:08, Hye-Shik Chang wrote:
> Hye-Shik Chang <hyeshik@gmail.com> added the comment:
> 
> Committed patch "cjkmactemporary.diff" as r65988 in the py3k branch.
> I'll open another issue for cjkcodecs implementation of Mac codecs.

Thanks !
msg72189 - (view) Author: John Smith (josm) Date: 2008-08-30 10:04
Confirmed now I can build Python on Mac OS X with Japanese locale.
I tested the followings. 

Python 2.6b3+ trunk:66060
Python 3.0b3+ py3k:66060

Thank you very much.
msg164148 - (view) Author: Olivier Berten (olivier-berten) Date: 2012-06-27 12:12
Any idea why mac cjk encodings still aren't available in Python 2.7 and 3.2 ?
History
Date User Action Args
2012-06-27 12:12:13olivier-bertensetversions: + Python 2.7, Python 3.2, - Python 2.6, Python 3.0
nosy: + ezio.melotti, olivier-berten

messages: + msg164148

components: - Build, Demos and Tools, Library (Lib), Macintosh
2008-08-30 10:04:56josmsetmessages: + msg72189
2008-08-25 15:32:31lemburgsetmessages: + msg71940
2008-08-23 08:08:29hyeshik.changsetstatus: open -> closed
resolution: fixed
messages: + msg71802
2008-08-20 06:09:34hyeshik.changsetfiles: + cjkmactemporary.diff
2008-07-08 12:32:38mokeefesetnosy: + mokeefe
2008-06-27 05:42:38hyeshik.changsetfiles: + maccjkcodecs-1-py3k.diff
2008-06-26 12:05:35hyeshik.changsetfiles: + maccjkcodecs-1-trunk.diff
messages: + msg68780
2008-02-25 06:14:48hyeshik.changsetassignee: lemburg -> hyeshik.chang
messages: + msg62974
nosy: + hyeshik.chang
2008-01-06 22:29:45adminsetkeywords: - py3k
versions: Python 2.6, Python 3.0
2007-11-20 00:50:18christian.heimessetpriority: normal
keywords: + py3k
2007-10-15 17:57:26lemburgsetmessages: + msg56456
versions: + Python 2.6
2007-10-15 17:56:26lemburgsetmessages: + msg56455
2007-10-15 17:27:30gvanrossumsetassignee: lemburg
messages: + msg56449
nosy: + gvanrossum, lemburg
2007-10-14 07:57:42georg.brandlsetversions: + Python 3.0
2007-10-13 22:43:19loewissetkeywords: + patch
2007-10-13 10:07:09josmcreate