classification
Title: OS X 10.6 IDLE, tkinter: Cocoa Tk 8.5 crash when composite character typed in text field
Type: crash Stage: resolved
Components: IDLE, Tkinter Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: benjamin.peterson, georg.brandl, loewis, michael.foord, naguilera, ned.deily, rhettinger, ronaldoussoren
Priority: release blocker Keywords:

Created on 2011-01-21 14:27 by naguilera, last changed 2011-01-25 22:08 by vstinner. This issue is now closed.

Messages (25)
msg126739 - (view) Author: Nestor Aguilera (naguilera) Date: 2011-01-21 14:27
When trying to type 'ñ' in idle 3.2 (no problem in terminal), python "quits unexpectedly" when started from terminal:

$ idle3
2011-01-21 11:21:55.883 Python[5228:a07] setCanCycle: is deprecated.  Please use setCollectionBehavior instead
2011-01-21 11:21:55.890 Python[5228:a07] setCanCycle: is deprecated.  Please use setCollectionBehavior instead
2011-01-21 11:21:55.891 Python[5228:a07] setCanCycle: is deprecated.  Please use setCollectionBehavior instead
2011-01-21 11:22:08.777 Python[5228:a07] An uncaught exception was raised
2011-01-21 11:22:08.779 Python[5228:a07] *** -[NSCFString characterAtIndex:]: Range or index out of bounds
2011-01-21 11:22:08.781 Python[5228:a07] *** Terminating app due to uncaught exception 'NSRangeException', reason: '*** -[NSCFString characterAtIndex:]: Range or index out of bounds'
*** Call stack at first throw:
(
	0   CoreFoundation                      0x00007fff829c47b4 __exceptionPreprocess + 180
	1   libobjc.A.dylib                     0x00007fff816300f3 objc_exception_throw + 45
	2   CoreFoundation                      0x00007fff829c45d7 +[NSException raise:format:arguments:] + 103
	3   CoreFoundation                      0x00007fff829c4564 +[NSException raise:format:] + 148
	4   Foundation                          0x00007fff839c15e1 -[NSCFString characterAtIndex:] + 97
	5   Tk                                  0x00000001012ba035 -[TKApplication(TKKeyEvent) tkProcessKeyEvent:] + 664
	6   Tk                                  0x00000001012c0b38 TkMacOSXEventsCheckProc + 349
	7   Tcl                                 0x0000000101175667 Tcl_DoOneEvent + 316
	8   _tkinter.so                         0x000000010077d801 Tkapp_MainLoop + 177
	9   Python                              0x00000001000b2b62 PyEval_EvalFrameEx + 30530
	10  Python                              0x00000001000b34ca PyEval_EvalCodeEx + 1770
	11  Python                              0x00000001000b19e3 PyEval_EvalFrameEx + 26051
	12  Python                              0x00000001000b1bfd PyEval_EvalFrameEx + 26589
	13  Python                              0x00000001000b34ca PyEval_EvalCodeEx + 1770
	14  Python                              0x00000001000b37df PyEval_EvalCode + 63
	15  Python                              0x00000001000da19b PyRun_FileExFlags + 187
	16  Python                              0x00000001000da469 PyRun_SimpleFileExFlags + 521
	17  Python                              0x00000001000ef253 Py_Main + 3059
	18  Python                              0x0000000100000e5f 0x0 + 4294970975
	19  Python                              0x0000000100000d04 0x0 + 4294970628
)
terminate called after throwing an instance of 'NSException'
Abort trap
 
#----------------------------------------------
The welcome in idle 3.2 is:

Python 3.2rc1 (r32rc1:88040, Jan 15 2011, 13:31:22) 
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "copyright", "credits" or "license()" for more information.

#----------------------------------------------
I cannot start IDLE by double-clicking its icon in the Finder.

Best,

Néstor Aguilera
msg126753 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-01-21 16:59
No problem on Linux (Debian Sid): I tried "ŁñØ=1" in IDLE interpreter (written using the compose key).

It looks like the bug is specific to Mac OS X and comes from Tk directly:
http://sourceforge.net/tracker/index.php?func=detail&aid=2907388&group_id=12997&atid=112997
"(...)  this crash happens for [all] composite characters. (...) the crash is caused by the "sticky" modifier key and not by the accented character itself."
msg126754 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-01-21 17:00
> I cannot start IDLE by double-clicking its icon in the Finder.

You may open a new issue for this proble.
msg126758 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-01-21 17:18
(Regarding the second problem: if IDLE does not launch when you double-click on it, please check for and report any error messages from /var/log/system.log at the time. You can use /Applications/Utilities/Console.app to view system.log.)
msg126762 - (view) Author: Nestor Aguilera (naguilera) Date: 2011-01-21 18:26
Thanks Victor and Ned, I'll send a report on the second issue as well (I thought it was known).

Néstor Aguilera
msg126797 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-01-21 23:39
As this clearly seems to be a Tk bug, I suggest to close this report as "won't fix - 3rd party".
msg126801 - (view) Author: Nestor Aguilera (naguilera) Date: 2011-01-21 23:56
On 21 Jan 2011, at 20:39, Martin v. Löwis wrote:

> Martin v. Löwis <martin@v.loewis.de> added the comment:
> 
> As this clearly seems to be a Tk bug, I suggest to close this report as "won't fix - 3rd party".

I see your point. The problem is that IDLE is somewhat included with Python (so in a sense it is not "3rd party"), and seems like a good tool for learning Python: my concern is similar to that in message 126276 <http://bugs.python.org/msg126276>.

All the best,

Nestor
msg126803 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-01-22 00:12
> I see your point. The problem is that IDLE is somewhat included with
> Python (so in a sense it is not "3rd party"), and seems like a good
> tool for learning Python: my concern is similar to that in message
> 126276 <http://bugs.python.org/msg126276>.

Unfortunately, there is little we can do about this (IIUC). One option
would be to include a fixed Tk version, but there is none to include
(IIUC). The only real solution to "fix" both the reported problem and
the problem of msg126276 would be to stop including IDLE with the MacOS
distributions, and asking users who want to use IDLE to switch to
Linux or Windows.
msg126805 - (view) Author: Nestor Aguilera (naguilera) Date: 2011-01-22 00:19
On 21 Jan 2011, at 21:12, Martin v. Löwis wrote:

> 
> Martin v. Löwis <martin@v.loewis.de> added the comment:
> 
>> I see your point. The problem is that IDLE is somewhat included with
>> Python (so in a sense it is not "3rd party"), and seems like a good
>> tool for learning Python: my concern is similar to that in message
>> 126276 <http://bugs.python.org/msg126276>.
> 
> Unfortunately, there is little we can do about this (IIUC). One option
> would be to include a fixed Tk version, but there is none to include
> (IIUC). The only real solution to "fix" both the reported problem and
> the problem of msg126276 would be to stop including IDLE with the MacOS
> distributions, and asking users who want to use IDLE to switch to
> Linux or Windows.

(:-)

I hope you will not consider to also stop including tkinter!

Nestor
msg126855 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-01-22 21:03
This seems to me to be a nasty release blocker. As documented in the Tk bug that Victor cited (thanks!), the crash occurs in Cocoa Tk when an input method prefix "dead key" combining character is typed in a Tk text field. So, for example, with the US Extended input method on a US keyboard, to type ñ, you can type the two keystrokes: Option-n n.  Or for an ä: Option-u a. Typing the leading dead key causes Tk to crash with the "index out of bounds" exception, bringing down tkinter and IDLE with it (and users losing their edits).  There is a workaround for users to enter these characters without a crash.  OS X also supports a postfix "dead key" combining sequence; ñ -> n Shift-Option-n and ä -> a Shift-Option-u.  The postfix forms do not cause a crash with Cocoa Tk 8.5 and the expected character is stored in the field, although the screen may not get properly updated.  But, I suspect of the two, the prefix method is much more commonly used and, in any case, it is not reasonable to warn users that they risk a crash if they enter a common keyboard sequence.  (The other common method to enter characters not available on the keyboard is to use the system Character Viewer which gives access to all Unicode code points.  Unfortunately, Character Viewer input doesn't seem to work with Cocoa Tk 8.5.  That would seem to be a major deficiency in Cocoa Tk's support for full Unicode input.)

This problem is exhibited in the Apple-supplied Cocoa Tk 8.5 shipped with OS X 10.6 and the Apple-supplied IDLEs (/usr/bin/idle2.6 and /usr/bin/idle2.5) suffer from the same vulnerability.  So this is not a new problem in the ActiveState Tk 8.5.9.  The Tk bug report on the SourceForge tracker was opened and accepted nearly a year ago and, although subsequent comments and activity have raised the priority to 9 (critical), there is no indication that a fix is forthcoming soon.  I am pinging both at the SF tracker and on the Tcl Mac list and will propose some alternatives for 3.2 here shortly.
msg126859 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-01-22 22:20
Here's a brain dump of possible options I see for 3.2rc2:

    1. document and do nothing
        -> IDLE (and other tkinter program) crashes whenever a user types a composing character (like Option u in US layout for an umlaut) when the 64-/32-bit installer is used

    2. don't ship the OS X 64-bit installer, only the traditional 32-bit one
        
    3a. don't ship IDLE.app and bin/idle with 64-bit installer
        
    3b. don't ship tkinter, IDLE.app, and bin/idle with 64-bit installer
        
    4. for 64-bit installer, link tkinter with Aqua Tk 8.4, like 32-bit installer
        -> no tkinter available in (default) 64-bit mode, users would need to force 32-bit execution
        -> need to fix IDLE to only run in 32-bit mode
        
    5a. for 64-bit installer, link tkinter with a third-party X11 Tk 8.5, most likely MacPorts
        -> X11 version works in both 64- and 32- and is likely more widely tested (on Unixes). MacPorts for example currently supplies an X11 Tk by default.
        -> non-native look and feel
        -> is X11 installed by default on 10.6? (check and document)
        -> MacPorts currently does not supply binary packages so users would need to install Xcode developer tools

    5b. for 64-bit installer, have the installer build an X11 Tk 8.5 (using the installer's existing third-party lib infrastructure) and ship with the 64-bit installer
        -> same as 5a except that IDLE and tkinter would now work out of the box but at the cost of the additional unknown development and testing work to get Tk built with the installer

At the moment, for 3.2rc2 my inclination is to pursue option 4 and at least explore the feasibility of option 5b with a fallback to option 3a while encouraging (but not depending) on a future fix for Cocoa Tk. Again, this applies only to the 64-/32-bit installer; the traditional 32-bit installer is linked with Carbon Tk 8.4 which does not have this problem.

Post 3.2, I think it makes sense to pursue a previously-discussed option to support multiple Tk versions with one release so users can choose.  That would also have the potential benefit of making Tk 8.5 available in one form or another for users of earlier OS X releases.  But that's a matter for later exploration.
msg126860 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-01-22 22:29
Hmm.  It seems better to me to accept this bug (and document it, and point out that it isn't Python's fault) than depriving 64-bit users of IDLE, or (even worse) of tkinter.

If you can manage a solution that doesn't remove IDLE, I'm in favor, but otherwise option 1. is my favorite.  I trust there is no possibility of a workaround from the Python or _tkinter side of things?
msg126861 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-01-22 22:54
Since the key strokes are captured directly by Tk, I doubt that anything could be worked around on the Python side unless there was some Tk option to disable a class of input characters or something. Seems unlikely but I can check with the tcl-mac list.

I should point out regarding 5a and 5b that with the X11 option (as with MacPorts today) using the Apple X11 Quartz window manager server (the default option) is a relatively seemless experience for the user. Clicking on IDLE.app or executing bin/idle in a shell causes the Apple X11.app to launch automatically and the expected IDLE windows appear alongside the user's existing (non-X11) windows.  The main difference is that there is the look of the windows and that the menu bars are in the X windows rather than in the normal OS X menu bar at the top of the screen (that menu bar is for X11.app).
msg126868 - (view) Author: Nestor Aguilera (naguilera) Date: 2011-01-23 01:10
Thanks Ned for thinking of ways out.

- If I had a choice, I would agree with Georg's choice (Ned's option 1): most users will not use the composite characters and need not go through complications, or worse, cannibalization of the distribution (3a and 3b are unacceptable).

Perhaps the existence of the bug could be documented together with the need of a proper Tcl/Tk version.

- My experience with python 3.1.3 with OS 10.6 is that when starting IDLE I always get several messages of the sort:

      Python(1305,0xa0a5c540) malloc: *** error for object
      0x1a277738: pointer being freed was not allocated
      *** set a breakpoint in malloc_error_break to debug

possibly leading to memory leaks. Thus, I am not happy with 4.

- As for 5a, perhaps with the warnings of Tcl/Tk and composite crashes, one could announce this alternative installation. 5b doesn't seem to be worth the trouble, given the existence of the MacPorts version.

My 2 cents.

Néstor
msg126870 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-01-23 01:59
> Hmm.  It seems better to me to accept this bug (and document it, and
> point out that it isn't Python's fault) than depriving 64-bit users
> of IDLE, or (even worse) of tkinter.

I disagree. There aren't really "64-bit users" on OSX, thanks to fat
binaries. So if starting IDLE would start a 32-bit interpreter, users
likely won't even notice. If they do notice, they can still run in
64-bit mode from the command line.

In my experience with fat python builds on OSX, it's typically better
to prefer 32-bit mode, anyway, as extension modules often don't come
in a fat version, but often in 32-bit mode only.
msg126877 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-01-23 07:33
> I disagree. There aren't really "64-bit users" on OSX, thanks to fat
> binaries. So if starting IDLE would start a 32-bit interpreter, users
> likely won't even notice. If they do notice, they can still run in
> 64-bit mode from the command line.

Okay, fair enough.  If it's easy to always let IDLE run in 32-bit mode, I'm fine with that.  What about other programs using tkinter?
msg126883 - (view) Author: Nestor Aguilera (naguilera) Date: 2011-01-23 11:35
On 23 Jan 2011, at 04:33, Georg Brandl wrote:

> Georg Brandl <georg@python.org> added the comment:
> 
>> I disagree. There aren't really "64-bit users" on OSX, thanks to fat
>> binaries. So if starting IDLE would start a 32-bit interpreter, users
>> likely won't even notice. If they do notice, they can still run in
>> 64-bit mode from the command line.

> Okay, fair enough.  If it's easy to always let IDLE run in 32-bit mode, I'm fine with that.  What about other programs using tkinter?

In a previous message (http://bugs.python.org/issue10973#msg126868) I mentioned that I get warning messages with the 32-bit version of 3.1.3 in OSX 10.6. Would a 32-bit version for 3.2 correct these?
msg126890 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2011-01-23 14:16
There are a few issues here.

X11 is not installed by *default* on Mac OS X (it is supplied separately) so it doesn't provide an "out of the box" solution.

Starting IDLE as 32bit alone doesn't solve the problem as it launches a subprocess and you have to ensure that is launched as a 32bit process as well.

I think *clearly* documenting that the Mac OS X 10.6 version *requires* Activestate Tcl / Tk 8.5 is the best solution (assuming I am correct in thinking that using the Activestate distribution solves the problem). This is really a duplicate of issue 10969 by the way. See the discussion there for suggested ways to clarify the dependency on the download page.
msg126897 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-01-23 18:49
Michael, the crash documented here is *not* fixed by installing ActiveState 8.5; it is currently a problem seen with both A/S and Apple's Tk 8.5.  Also, to answer my own question, I double-checked the Snow Leopard 10.6 installation DVD and, unlike earlier releases, X11 is now installed by default.

That said, there is now some action on the Tcl front which may result in an official patch to at least prevent the crash.  If so, there may be yet another option:

6. for 64-bit installer, have the installer build a Cocoa Tk 8.5 (using the installer's existing third-party lib infrastructure) and ship with the 64-bit installer

So, at the moment, I think the best course of action is to explore the various patch options and, with more info available, a decision for 3.2rc2 made prior to cutoff.
msg126942 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2011-01-24 19:24
Activestate has said (replying to me on Twitter as it happens) that a patch is available and they will do a new 8.5 release before 3.2 final.
msg126957 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-01-24 21:17
Yes, see http://permalink.gmane.org/gmane.comp.lang.tcl.mac/6871

So the plan for 3.2 is to assume there will be an updated ActiveState Tk 8.5 available.  I will update the web status page accordingly.  And this issue can be closed as a third-party library issue.
msg127000 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2011-01-25 12:32
Michael: Why must the subprocess started by IDLE be in 32-bit mode if we'd run IDLE in 32-bit mode?  AFAIK there is no technical reason to do so.

Not that running IDLE in 32-bit mode is an option, it wouldn't fix the issue by itself unless we'd link Tkinter to Tk 8.4 and that would make it impossible to use Tkinter in 64-bit mode at all.

BTW. I'm -1 on using the X11 version of Tk, that would make IDLE look even less like a real Mac application. It would also require additional work to ensure that IDLE.app keeps working as expected.
msg127003 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2011-01-25 12:41
Ronald: The subprocess also uses Tkinter (right?) so would also require 32bit. 

FWIW I'm -1 on X11 as well.
msg127013 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2011-01-25 14:42
On 25 Jan, 2011, at 13:41, Michael Foord wrote:

> 
> Michael Foord <michael@voidspace.org.uk> added the comment:
> 
> Ronald: The subprocess also uses Tkinter (right?) so would also require 32bit. 

Not AFAIK.  I'm pretty sure I've had a 3-way build in the past where IDLE ran as a 32-bit binary and the subprocess was 64-bit one on supported systems.
msg127014 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2011-01-25 14:47
The reason I think it is an issue is that a previous release of Python 2.7 could start IDLE (the initial window would appear), but a dialog would also appear saying that it could not connect to the subprocess and IDLE would exit. IDLE itself had been set to run as a 32bit process by default, but "various people" thought the issue was caused by the subprocess not being launched as 32bit.
History
Date User Action Args
2011-02-10 13:36:50r.david.murraylinkissue11170 superseder
2011-01-25 22:08:59vstinnersetnosy: - vstinner
2011-01-25 14:47:20michael.foordsetnosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg127014
2011-01-25 14:42:45ronaldoussorensetnosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg127013
title: OS X 10.6 IDLE, tkinter: Cocoa Tk 8.5 crash when composite character typed in text field -> OS X 10.6 IDLE, tkinter: Cocoa Tk 8.5 crash when composite character typed in text field
2011-01-25 12:41:14michael.foordsetnosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg127003
2011-01-25 12:32:14ronaldoussorensetnosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg127000
2011-01-24 21:17:07ned.deilysetstatus: open -> closed
nosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg126957

resolution: wont fix
stage: needs patch -> resolved
2011-01-24 19:24:03michael.foordsetnosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg126942
2011-01-23 18:49:56ned.deilysetnosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg126897
2011-01-23 14:16:37michael.foordsetnosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg126890
2011-01-23 11:35:54naguilerasetnosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg126883
title: OS X 10.6 IDLE, tkinter: Cocoa Tk 8.5 crash when composite character typed in text field -> OS X 10.6 IDLE, tkinter: Cocoa Tk 8.5 crash when composite character typed in text field
2011-01-23 07:33:31georg.brandlsetnosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg126877
2011-01-23 01:59:32loewissetnosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg126870
title: OS X 10.6 IDLE, tkinter: Cocoa Tk 8.5 crash when composite character typed in text field -> OS X 10.6 IDLE, tkinter: Cocoa Tk 8.5 crash when composite character typed in text field
2011-01-23 01:10:07naguilerasetnosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg126868
2011-01-22 22:54:53ned.deilysetnosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg126861
2011-01-22 22:29:54georg.brandlsetnosy: loewis, georg.brandl, rhettinger, ronaldoussoren, vstinner, benjamin.peterson, ned.deily, michael.foord, naguilera
messages: + msg126860
2011-01-22 22:20:28ned.deilysetnosy: + ronaldoussoren, rhettinger, michael.foord

messages: + msg126859
stage: needs patch
2011-01-22 21:03:36ned.deilysetpriority: normal -> release blocker


components: + Tkinter
title: 'ñ' not working with IDLE 3.2rc1 - OSX 10.6.6 -> OS X 10.6 IDLE, tkinter: Cocoa Tk 8.5 crash when composite character typed in text field
nosy: + benjamin.peterson, georg.brandl
versions: + Python 2.7
messages: + msg126855
2011-01-22 00:19:20naguilerasetmessages: + msg126805
2011-01-22 00:12:46loewissetmessages: + msg126803
2011-01-21 23:56:54naguilerasetmessages: + msg126801
2011-01-21 23:39:11loewissetnosy: + loewis
messages: + msg126797
2011-01-21 18:26:28naguilerasetmessages: + msg126762
2011-01-21 17:18:09ned.deilysetassignee: ned.deily
messages: + msg126758
2011-01-21 17:00:20vstinnersetmessages: + msg126754
2011-01-21 16:59:35vstinnersetmessages: + msg126753
2011-01-21 14:36:28pitrousetnosy: + ned.deily
2011-01-21 14:31:39vstinnersetnosy: + vstinner
2011-01-21 14:27:36naguileracreate