classification
Title: IDLE does not work with Unicode
Type: behavior Stage: resolved
Components: IDLE, Unicode Versions: Python 3.1
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: asvetlov Nosy List: asvetlov, gpolo, jchelary, ned.deily, weirdink13, zzyzx
Priority: normal Keywords:

Created on 2008-12-13 07:43 by zzyzx, last changed 2012-03-23 15:03 by weirdink13. This issue is now closed.

Messages (11)
msg77717 - (view) Author: Aki Wakabayashi (zzyzx) Date: 2008-12-13 07:43
I have installed python 3.0 on Ubuntu 8.10 yesterday and played around
with the new unicode features and had no problems with Japanese
characters(both in interactive and script mode). However, after
rebooting, IDLE will no longer let me input any Japanese characters.
(I have it set to UFT-8, if it makes any difference anymore)

The terminal works fine. It lets me input Japanese and runs commands
correctly.

So I used gedit and saved a simple print("Hello(in Japanese chars)"),
and run it with IDLE 3.0 but I get a blank line >>>. 

(Can I also make sure that I'm opening IDLE correctly? This is how I do it:
1)Open Terminal
2)~$ Python.3.0
3)>>> input idlelib.idle 

Thank You in advance.
msg91214 - (view) Author: Guilherme Polo (gpolo) * (Python committer) Date: 2009-08-03 03:20
I can't seem to reproduce that, maybe it could be a tk issue ?

Can you try writing anything (that doesn't work on IDLE) on a
tkinter.Text widget to see if it shows there ? You could use this code
below:

from tkinter import Text

text = Text()
print(text.tk.call('info', 'patchlevel'))
text.focus_set()
text.pack()
text.mainloop()
msg94884 - (view) Author: Jean-Christophe Helary (jchelary) Date: 2009-11-04 09:52
I've installed Python 3.1.1 on OSX today.
1) When I use the Japanese input from OSX, IDLE interprets any character 
I attempt to type as a space.
2) When I paste a Japanese string from a different place, it is 
correctly handled. For ex:
>>> print('ここ')
ここ
>>> 

While on Python 2.5's IDLE I had:
>>> print('ここ')
Unsupported characters in input

>>> 

by default.

In any case, IDLE 3.1.1 does not respect the input source and that makes 
it impossible to directly type Japanese (if not other double byte 
character based languages).

Note: IDLE respect OSX dead keys and I can correctly use "Alt+c" to 
input "ç".
msg156510 - (view) Author: Daniel Swanson (weirdink13) Date: 2012-03-21 20:00
I opened my IDLE (v. 3.2.2 windows xp) and pasted in
print('ここ')
it printed
ここ
just fine.
msg156511 - (view) Author: Daniel Swanson (weirdink13) Date: 2012-03-21 20:02
alt-c does nothing for me
msg156512 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2012-03-21 20:17
For now unicode BMP has full support in TK while non-BMP characters doesn't works.

'こ' character is BMP symbol:
>>> hex(ord('こ'))
'0x3053'
which is lesser than non-BMP space (starting from 0x10000).

I have no idea why alt-c doesn't converted to 'ç' (also BMP by the way) and why it has been processed by Helary's IDLE. I have no any Mac box nearby to check. 
Sure, 'alt-c' is minor problem if this is problem at all.
msg156514 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2012-03-21 20:22
I close this issue because:
- current 3.2 and upcoming 3.3 support Japanese characters very well.
- there are problems with non-BMP characters not supported currently but it's another issue.

See progress of #14200 and others for non-BMP.
msg156515 - (view) Author: Daniel Swanson (weirdink13) Date: 2012-03-21 20:25
I would say that alt-c is not a problem at all, but, some people might use 'ç' more that me, (I never have used 'ç' spesificaly)
msg156516 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2012-03-21 20:29
To add to the other comments, problems with input methods using Python 3 and Tkinter or IDLE are usually platform-specific issues with the implementation of Tk.  In particular, the issue Jean-Christophe reported with Python 3.1.1 was very likely due to its use of the old Tk 8.4 (at least with the python.org installer).  Current versions of Python 3.2.x for OS X 10.6+ link with the newer Tk 8.5, the very latest releases of which by ActiveState contain important fixes for input methods using composite characters (http://www.python.org/download/mac/tcltk/ has the most up-to-date information).  Also, note to the original poster, you either made a typo in the bug report ("UFT-8" - it should be "UTF-8") or, if you actually tried that locale, it might explain why you had problems.  If someone can reproduce a problem with a current Python 3.2.x or later, please re-open with details.
msg156517 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2012-03-21 20:39
Daniel Swanson, maybe my msg156512 was not obvious.
You can use 'ç' without any problem in IDLE — it is BMP character.

See http://en.wikipedia.org/wiki/Mapping_of_Unicode_characters
for details.
tkinter has full support for 'Basic Multilingual Plane' while last planes still has a problems.
msg156667 - (view) Author: Daniel Swanson (weirdink13) Date: 2012-03-23 15:03
ok
History
Date User Action Args
2012-03-23 15:03:38weirdink13setmessages: + msg156667
2012-03-21 20:39:25asvetlovsetmessages: + msg156517
2012-03-21 20:29:56ned.deilysetnosy: + ned.deily
messages: + msg156516
2012-03-21 20:25:28weirdink13setmessages: + msg156515
2012-03-21 20:22:40asvetlovsetstatus: open -> closed
messages: + msg156514

assignee: asvetlov
resolution: works for me
stage: test needed -> resolved
2012-03-21 20:17:02asvetlovsetmessages: + msg156512
2012-03-21 20:02:03weirdink13setmessages: + msg156511
2012-03-21 20:00:57weirdink13setnosy: + weirdink13
messages: + msg156510
2012-03-21 19:40:43asvetlovsetnosy: + asvetlov
2009-11-04 09:52:11jchelarysetnosy: + jchelary

messages: + msg94884
versions: + Python 3.1, - Python 3.0
2009-08-03 03:20:48gpolosetmessages: + msg91214
2009-04-26 22:15:52ajaksu2setnosy: + gpolo
priority: normal
components: + Unicode
type: performance -> behavior
stage: test needed
2008-12-13 07:43:48zzyzxcreate