classification
Title: IDLE: work around shortcuts bug in Windows' IMEs and tk
Type: behavior Stage: test needed
Components: IDLE Versions: Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: Constantine Ketskalo, louielu, serhiy.storchaka, terry.reedy
Priority: normal Keywords:

Created on 2017-08-21 12:03 by Constantine Ketskalo, last changed 2017-08-28 04:05 by Constantine Ketskalo.

Messages (14)
msg300623 - (view) Author: Constantine Ketskalo (Constantine Ketskalo) Date: 2017-08-21 12:03
Hi there.
Thank you for Python, I love this programming language.

I've found a bug with shortcuts. I'm from Ukraine, so I'm working with English, Ukrainian and Russian languages. But any shortcuts, where alphabet is involved (like Ctrl+Z, Ctrl+F, Ctrl+V and so on) work only when English is turned on. This is quite inconvenient. It's possible to work with it, but still confusing, because you can select something in IDLE, copy it with shortcut Ctrl+C, then try to paste it to another place and then you find out that it wasn't copied, because active language is Ukrainian. So you need to switch to English and repeat everything again.

I'm currently working with Python 3.6.1, Windows 8.1 Embedded enterprize x64.
Also I've noticed the same under Windows 7, Windows 10 in different versions of Python.
I believe there where Python 2.7, 2.11, 3.4, 3.5. Not sure about all the versions exectly, but for sure shortcuts worked the same in all the versions I've ever tried to use.

I hope this was helpful and I'm sure many developers, that work not only with English would be really happy if you would fix this.

Thanks again for your time and effort.
Best regards, Constantine.
msg300624 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-08-21 13:43
IDLE and Python/tkinter have nothing to do with the handling of keystrokes and their direct interpretation.  The behavior you describe, the Ctrl key being disabled in Ukranian mode, strikes me as a bug in either Windows or the Input Method you use, or, less likely, the tcl tk GUI framework.  Someone who uses the Japanese IME with IDLE thinks Ctrl continues to work right in Japanese mode, but will check and report to me.

IDLE offers menu entries for most shortcuts.  Indeed, shortcuts are shortcuts relative to using the menu.  Cut, copy, and paste are on the Edit menu.  It should be accessible with Alt shortcuts.  C, C, and P are also on the right-click context menu.
msg300667 - (view) Author: Constantine Ketskalo (Constantine Ketskalo) Date: 2017-08-22 03:44
You must be kidding me? I know that it's possible to use it via menu. And my speed of software development will be 3 times slower. Great!
If this was a bug in Windows then why the heck it works perfectly in any other program including Python when I switch language to English?
msg300682 - (view) Author: Louie Lu (louielu) * Date: 2017-08-22 09:20
Hi, so you are using Windows built-in IEM with Ukrainian input method. And in this situation, you cannot use Ctrl+Z, Ctrl+V ..etc. right?
msg300685 - (view) Author: Constantine Ketskalo (Constantine Ketskalo) Date: 2017-08-22 11:50
Louie Lu, yes, I'm using Windows. But, sorry, what is IEM? I'm currently using native Python IDLE. Right click to .py file and "Edit with IDLE". However I'm starting to think about switching to Visual Studio, even though it's huge and slower...
Yes, I'm using both latin and cyrillic characters, and if just finished typing some string in Ukrainian, combinations Ctrl + Z/X/C/V/F/G/H/S, also Alt + G: they just don't work. Only if I switch back to English.
msg300687 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-08-22 12:27
IDLE's involvement with cut, copy, and paste is to tell the tcl/tk GUI framework what keys to associate with the operations. The keys depend on the OS-specific defaults or the keyset chosen by the user.  The actual operation on the Text widget is carried out by tk, in response to events it gets from the OS.

IME = Input Method Editor.  I just watched ^C and ^V work on Windows with Window's Japanese IME.  I properly closed this as an IDLE issue.  Serhiy, do you know anything about Windows' built-in Cyrillic IMEs and control key combinations?
msg300693 - (view) Author: Constantine Ketskalo (Constantine Ketskalo) Date: 2017-08-22 14:06
Terry J. Reedy, Japaneese has no cyrillic characters, it's alphabet consists of hieroglyphs and it has nothing to do with Ukrainian even if it looks the same for you.
"about Windows' built-in Cyrillic IMEs and control key combinations?"
In depth - no. I'm just using them and I know that Python is the only program I've ever met that doesn't exept same combinations with Ukrainian or Russian language on.
If you want to test what I'm talking about - just temporary add to your languages on your PC Ukrainian language, open Python IDLE and try any shortcuts while input language is Ukrainian.
Cyrillic characters can have sometimes problems with encoding, because they are not included to default ASCII table, only to extended versionis. Could that be a reason?
Or I just don't understand something? If not, please explain or give me a link or tip where can I find out what's goind on and who can fix it. Thanks.
msg300694 - (view) Author: Constantine Ketskalo (Constantine Ketskalo) Date: 2017-08-22 14:08
*versionis = versions
msg300716 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-08-22 17:58
I confirm the issue. Seems this is a particularity of Tk implementation on Windows (it looks as a bug from user's point of view). There is a workaround, but it is too complex. For the particular keyboard layout and for every registered shortcut you need to find the character emitted by pressing the corresponding key, find its code in the current ANSI codepage, find the name for this code in the keysym table [1], and register additional shortcut for this name. For example for making Ctrl-O working with Ukrainian encoding you need to add <Control-Key-ugrave> open-window-from-file action. Cyrillic letter "щ" is emitted when the key "o" is pressed in Ukrainian layout, it has the code 249 in CP-1251, which corresponds the name ugrave in the keysym table. All this is too complex for IDLE. I think that we should close this issue as "Won't fix" and hope that this will be fixed automatically in future versions of Tk.

Firefox starved from similar issue for many years. It was fixed in recent versions. But Firefox is much more complex program than IDLE.

[1] https://www.tcl.tk/man/tcl/TkCmd/keysyms.htm
msg300737 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-08-23 01:12
Constantine: the point of checking the behavior of the Japanese IME was to determine whether there is a problem with all IMEs or just some.  Since Ctrl + the 2nd key from the left on the bottom row (marked 'X') invokes 'Delete' in both American and Japanese entry modes, the behavior you want should be possible.

I do not know where the bug is, and do not care, other than knowing that it is not in IDLE (or Python).  I do care that anyone in the world, like you, with sufficient knowledge to use Python, be able to also use IDLE if they chose.   Even if switching keyboards with alt-shift is a fast, habituated, motion, it is clearly a nuisance in this context  So, even though IDLE is not Firefox, I am willing to consider somehow implementing what Serhiy described and supporting either adding workaround keyset(s) to IDLE or make it easier for users to create their own custom keysets that work.

When I mentioned the menus, I assume that the alt menu shortcuts work, so alt-e, alt-c, for instance, would copy a selection.  When we tried this with the Japanese keyboard, the alt shortcuts work for the top menu (F_ile, E_dit, S_hell, etc) but not for the drop-down submenus.  After alt-e displays the edit menu, alt-c, etc, have no apparent effect.  Another bug, and one which I doubt we can work around without language-specific menus.  (This is a separate issue.)

Step 1 for this issue: what is the scope of the problem.  I presume it is not unique to Ukranian.  Does it affect all Cyrillic IMEs?  If so, do different cyrillic keyboards have the the alphabetic chars in different places, so that a separate workaround is needed for every IME, even for letter chars?
msg300745 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-08-23 07:54
Yes, it affects all Cyrillic layouts, as well as Greek and other alphabetical systems. It also affects layouts for Western-European languages with keys for accented characters. Different Cyrillic keyboards have different characters on the same key, but I don't know if they have different keys for the same character. Fortunately all Cyrillic languages use the same ANSI encoding. But I think there are conflicts between Cyrillic and non-Cyrillic layouts. Different keys on different keyboards can produce the same code (which means different characters in different codepages).
msg300746 - (view) Author: Constantine Ketskalo (Constantine Ketskalo) Date: 2017-08-23 08:20
Bug is not in Python? Then where? Windows?

- I assume that the alt menu shortcuts work...
- NO! And I've written that before as well! Alt + combinations don't work neither with 

cyrillic!

- Does it affect all Cyrillic IMEs?
- Yes, I've written that before already - shortcuts don't work neither with Ukrainian, nor 

with Russian
- If so, do different cyrillic keyboards have the the alphabetic chars in different 

places, so that a separate workaround is needed for every IME, even for letter chars?
- Yes and no. Most letters in different cyrillic alphabets are the same, so they have the 

same place on the keyboard(as well as the same encoding), but some of them are different, 

so in different IMEs same key gives different letter.
For instance English 's' in Ukrainian is 'і' and in Russian is 'ы'. But English 'c' is in 

both Ukrainian and Russian 'с' (looks the same, has different encoding then English, but 

has same encoding for Ukrainian and Russian).
msg300855 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-08-25 19:41
I obtained more info on the Japanese IME.  It has 2 modes of operation: 'romanji' and 'kana'.  In romanji mode, one types the ascii transliteration of the syllabic chars.  When one types type the syllable vowel , the program replaces the sequence of 1 to 3 ascii chars with a Japanese char.  So while typing 'akitsu', the 'a', 'ki', and 'tsu' would become 3 chars, with the 'k' and 'ts' momentarily visible.  Modifier combinations naturally work since the keyboard is being interpreted as outputting ascii chars.  In kana mode, japanese chars appear directly, but Ctrl must, in effect, temporarily revert the keys to their ascii interpretations.

A Chinese keyboard with a pinyan (romanization) input mode might do the same.  But non-English keyboards having an ascii mode is likely exceptional.  Given this and the two answers above, I conclude that delivering IDLE with a 'complete' set of IME-compatible keysets is an impossible problem.

I definitely want to document the problem.

I will stay open to the possibility of a 'customizer' that would prompt a user to hit all the character keys in a defined order and then augment an existing keyset using the method described in msg300716.  The table at https://www.tcl.tk/man/tcl/TkCmd/keysyms.htm suggests that this could potentially cover accented latin, cyrillic, greek, hebrew, arabic, and japanese keyboards.
msg300934 - (view) Author: Constantine Ketskalo (Constantine Ketskalo) Date: 2017-08-28 04:05
Impossible... Yea, right...
Will documenting it help to solve it?
If that's impossible, then how do other programs deal with it? What does make Python so special?
I still don't see anything impossible. If other programs can do it but Python IDLE can't do it for now, then it's called either "hard" or "I don't want to do it", but not "impossible".

First of all I would try to find out how does cyrillic work with other programs(sorry, I don't know that). Probably their approach to this situation could be just copied to Python IDLE behaviour.

Japanese wan't help you with cyrillic. This ticket is about cyrillic!!!

ascii is not the only encoding, that exists. Could it be as possible solution to use unicode incoding with Python when it comes down to hotkeys?

If there is something that makes Japaneese and cyrillic languages incompatible with each other to use hotkeys, well, then you could just add option to choose which language is preferable to user (either in installed program, or during installatioin).

Windows command prompt doesn't support cyrillic by default, but there is a code in C, that makes it work with cyrillic input and output. Python was written on C, so I'm sure it could be helpful even if it doesn't work directly.
Here is this code for C/C++:
`
SetConsoleCP(1251);
SetConsoleOutputCP(1251);
setlocale(0, "Ukrainian");
`
History
Date User Action Args
2017-08-28 04:05:09Constantine Ketskalosetmessages: + msg300934
2017-08-25 19:41:25terry.reedysetmessages: + msg300855
2017-08-23 08:20:35Constantine Ketskalosetmessages: + msg300746
2017-08-23 07:54:53serhiy.storchakasetmessages: + msg300745
2017-08-23 01:12:12terry.reedysetassignee: terry.reedy
components: + IDLE
title: Shortcuts bug in Python IDLE -> IDLE: work around shortcuts bug in Windows' IMEs and tk
resolution: third party ->
versions: + Python 3.7
messages: + msg300737
stage: resolved -> test needed
2017-08-22 17:58:01serhiy.storchakasetmessages: + msg300716
2017-08-22 14:08:23Constantine Ketskalosetmessages: + msg300694
2017-08-22 14:06:54Constantine Ketskalosetmessages: + msg300693
2017-08-22 12:27:29terry.reedysetcomponents: - IDLE
2017-08-22 12:27:14terry.reedysetassignee: terry.reedy -> (no value)

messages: + msg300687
nosy: + serhiy.storchaka
2017-08-22 11:50:23Constantine Ketskalosetmessages: + msg300685
2017-08-22 09:20:16louielusetnosy: + louielu
messages: + msg300682
2017-08-22 03:44:32Constantine Ketskalosetstatus: closed -> open

messages: + msg300667
2017-08-21 13:43:37terry.reedysetstatus: open -> closed
resolution: third party
messages: + msg300624

stage: resolved
2017-08-21 12:03:49Constantine Ketskalocreate