classification
Title: IDLE 3.x can crash decoding recent file list
Type: crash Stage: resolved
Components: IDLE Versions: Python 3.1, Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: georg.brandl, naguilera, ned.deily
Priority: high Keywords: patch

Created on 2011-01-21 18:40 by naguilera, last changed 2011-01-24 22:28 by ned.deily. This issue is now closed.

Files
File name Uploaded Description Edit
issue10974.patch ned.deily, 2011-01-24 03:06
Messages (6)
msg126764 - (view) Author: Nestor Aguilera (naguilera) Date: 2011-01-21 18:40
Here is the report from "Console" (date/time removed):

[0x0-0x1a11a1].org.python.IDLE[5541]	Traceback (most recent call last):
[0x0-0x1a11a1].org.python.IDLE[5541]	  File "/Applications/Python 3.2/IDLE.app/Contents/Resources/idlemain.py", line 73, in <module>
[0x0-0x1a11a1].org.python.IDLE[5541]	    main()
[0x0-0x1a11a1].org.python.IDLE[5541]	  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/PyShell.py", line 1388, in main
[0x0-0x1a11a1].org.python.IDLE[5541]	    shell = flist.open_shell()
[0x0-0x1a11a1].org.python.IDLE[5541]	  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/PyShell.py", line 277, in open_shell
[0x0-0x1a11a1].org.python.IDLE[5541]	    self.pyshell = PyShell(self)
[0x0-0x1a11a1].org.python.IDLE[5541]	  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/PyShell.py", line 813, in __init__
[0x0-0x1a11a1].org.python.IDLE[5541]	    OutputWindow.__init__(self, flist, None, None)
[0x0-0x1a11a1].org.python.IDLE[5541]	  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/OutputWindow.py", line 16, in __init__
[0x0-0x1a11a1].org.python.IDLE[5541]	    EditorWindow.__init__(self, *args)
[0x0-0x1a11a1].org.python.IDLE[5541]	  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/EditorWindow.py", line 267, in __init__
[0x0-0x1a11a1].org.python.IDLE[5541]	    self.update_recent_files_list()
[0x0-0x1a11a1].org.python.IDLE[5541]	  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/EditorWindow.py", line 777, in update_recent_files_list
[0x0-0x1a11a1].org.python.IDLE[5541]	    rf_list = rf_list_file.readlines()
[0x0-0x1a11a1].org.python.IDLE[5541]	  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/encodings/ascii.py", line 26, in decode
[0x0-0x1a11a1].org.python.IDLE[5541]	    return codecs.ascii_decode(input, self.errors)[0]
[0x0-0x1a11a1].org.python.IDLE[5541]	UnicodeDecodeError: 'ascii' codec can't decode byte 0xcc in position 55: ordinal not in range(128)
com.apple.launchd.peruser.501[168]	([0x0-0x1a11a1].org.python.IDLE[5541]) Exited with exit code: 1

Thanks,

Néstor Aguilera
msg126821 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-01-22 07:50
From the traceback, IDLE is crashing on a decode error when trying to process the list of recently used files which is used to populate its recent files menu.  That list of files is saved in ~/.idlerc/recent-file.lst so an easy workaround is to delete the file before launching IDLE.app.  Most likely, the list of files contains at least one path name with a non-ASCII character.  IDLE appears to be dependent here on locale LANG settings.  When launched by double-clicking, like all Mac GUI apps IDLE.app does not inherited environment variables set in shell profiles, as in a terminal session.  The problem can be easily reproduced with bin/idle which is started from a shell environment:

$ echo $LANG
en_US.UTF-8
$ echo "/Users/nad/Documents/test_ä.py" > ~/.idlerc/recent-files.lst
$ /usr/local/bin/idle3.2
# with UTF-8 locale, idle starts normally with the umlauted file name in the recent files menu
$ unset LANG
$ /usr/local/bin/idle3.2
Traceback (most recent call last):
  File "/usr/local/bin/idle3.2", line 5, in <module>
    main()
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/PyShell.py", line 1388, in main
    shell = flist.open_shell()
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/PyShell.py", line 277, in open_shell
    self.pyshell = PyShell(self)
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/PyShell.py", line 813, in __init__
    OutputWindow.__init__(self, flist, None, None)
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/OutputWindow.py", line 16, in __init__
    EditorWindow.__init__(self, *args)
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/EditorWindow.py", line 267, in __init__
    self.update_recent_files_list()
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/EditorWindow.py", line 777, in update_recent_files_list
    rf_list = rf_list_file.readlines()
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 26: ordinal not in range(128)

A similar exception occurs (without a crash) when saving a file to a non-ASCII file name and LANG is not properly set:

Exception in Tkinter callback
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/tkinter/__init__.py", line 1399, in __call__
    return self.func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/MultiCall.py", line 166, in handler
    r = l[i](event)
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/IOBinding.py", line 328, in save
    self.save_as(event)
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/IOBinding.py", line 350, in save_as
    self.updaterecentfileslist(filename)
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/IOBinding.py", line 515, in updaterecentfileslist
    self.editwin.update_recent_files_list(filename)
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/idlelib/EditorWindow.py", line 795, in update_recent_files_list
    rf_file.writelines(rf_list)
UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 16: ordinal not in range(128)
msg126828 - (view) Author: Nestor Aguilera (naguilera) Date: 2011-01-22 10:23
Ned: thanks for the explanation.

Nestor

=======================================================================

On 22 Jan 2011, at 04:50, Ned Deily wrote:

> 
> Changes by Ned Deily <nad@acm.org>:
> 
> 
> ----------
> Removed message: http://bugs.python.org/msg126820
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue10974>
> _______________________________________
msg126885 - (view) Author: Nestor Aguilera (naguilera) Date: 2011-01-23 11:40
On 22 Jan 2011, at 04:50, Ned Deily wrote:

[...]
> A similar exception occurs (without a crash) when saving a file to a non-ASCII file name and LANG is not properly set:
[...]

Is there a way of telling tkinter to set the encoding to utf-8 (or whatever) before seeking recent-files.lst?
msg126914 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-01-24 03:06
IDLE 3.x currently does not specify an explicit encoding when reading or writing the recent files list (~/.idlerc/recent-files.lst) and it defaults to "errors=strict".  So IDLE 3.x is vulnerable to crashes if any of the recent files contain characters that can not be decoded using the LANG environment that IDLE is running under.  That can happen if the user changes LANG or the recent-files.lst was updated by IDLE 2.x or, on OS X, when running as IDLE.app.  The solution in the attached patch is to explicitly force the encoding of the file paths to utf_8 and to set "errors=replace".

Since this has the potential to crash IDLE, I propose it for inclusion in 3.2rc2.
msg126963 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-01-24 22:28
Committed in py3k (r88174) for 3.2rc2, with release manager approval, and in 3.1 (r88177).
History
Date User Action Args
2011-01-24 22:28:25ned.deilysetstatus: open -> closed

messages: + msg126963
resolution: fixed
stage: commit review -> resolved
2011-01-24 03:06:59ned.deilysetfiles: + issue10974.patch

title: IDLE 3.2 not loading on double-click in Finder, OSX 10.6 -> IDLE 3.x can crash decoding recent file list
keywords: + patch
nosy: + georg.brandl

messages: + msg126914
stage: needs patch -> commit review
2011-01-23 11:40:22naguilerasetmessages: + msg126885
2011-01-22 10:23:38naguilerasetmessages: + msg126828
2011-01-22 07:50:11ned.deilysetmessages: - msg126820
2011-01-22 07:50:02ned.deilysetmessages: + msg126821
2011-01-22 07:45:39ned.deilysetpriority: normal -> high

assignee: ned.deily
versions: + Python 3.1
nosy: + ned.deily

messages: + msg126820
stage: needs patch
2011-01-21 18:40:39naguileracreate