classification
Title: Wrong encoding in files saved from IDLE (3.0rc2 on Windows)
Type: behavior Stage:
Components: Versions: Python 3.0
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, andre, haypo, loewis
Priority: normal Keywords: patch

Created on 2008-11-14 11:03 by andre, last changed 2008-12-29 18:47 by loewis. This issue is now closed.

Files
File name Uploaded Description Edit
idle_dialog.png haypo, 2008-11-14 11:17
idle_remove_coding_dialog.py haypo, 2008-11-14 11:18
idle_coding_dialog_cancel.patch haypo, 2008-11-14 11:29
remove_coding_option.py loewis, 2008-11-19 16:57
Messages (8)
msg75855 - (view) Author: André (andre) Date: 2008-11-14 11:03
When saving a source file with non-ascii characters from an IDLE window,
on Windows platform (XP and Server 2003 at least)
with locale English US  

  locale.getdefaultlocale()
  ('en_US', 'cp1252')

IDLE prompts in IOBinding.py with the message 
"non Ascii found, yet no encoding declared, Add a line like # -*- 
coding: cp1252 -*-"

If accepted, the file is saved with the wrong encoding.
Afterwards, it is read back by IDLE without any problem and it looks 
good in the IDLE window.

However, if a string with non-ascii characters is sent to another 
module (i.e. email  MIMEText)
the string is wrong.

The same source would be rejected by python in terminal mode and by 
IDLE on Linux.
msg75856 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2008-11-14 11:17
> If accepted, the file is saved with the wrong encoding

"accept" means click on the "OK" button. You have to click on the 
biggest button "Edit my file" to create a valid file.

The problem here is the text in the dialog but also the possibility to 
create an invalid file. 

Two solutions:
 - don't ask user for a confirmation (always insert the header)
 - replace "OK" button by "Cancel" button: the user wants maybe an 
ASCII only file but inserted by error an unicode character (eg. non 
breaking space!)
msg75857 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2008-11-14 11:18
Quick but ugly solution: force the #coding: header without asking the 
user for that.
msg75858 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2008-11-14 11:29
Implementation of the second solution: replace Ok button by a Cancel 
button. Escape key is Cancel, whereas Return key is the default 
action: edit the file.
msg75869 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-11-14 16:16
The entire dialog should go away. The default source encoding is UTF-8
in Python 3.0, and IDLE should know that. The locale's encoding
shouldn't ever be considered.
msg76057 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-11-19 16:57
Here is a patch that removes the entire IDLE coding option machinery,
thus implementing PEP 3120.
msg76215 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-11-21 22:53
The patch is good and does simplify things: the py3k conversion to 
unicode may have some advantages sometimes!

I suggest to also remove the remaining two occurrences of 
"frameEncoding" in configDialog.py. This will suppress the extra space 
between the last two blocks of the dialog.
msg78474 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-12-29 18:47
Thanks for the review. Committed (with the proposed change) as r68022
and r68023.
History
Date User Action Args
2008-12-29 18:47:20loewissetstatus: open -> closed
resolution: fixed
messages: + msg78474
2008-11-21 22:53:17amaury.forgeotdarcsetkeywords: - needs review
nosy: + amaury.forgeotdarc
messages: + msg76215
2008-11-19 16:57:08loewissetkeywords: + needs review
files: + remove_coding_option.py
messages: + msg76057
2008-11-14 16:16:15loewissetnosy: + loewis
messages: + msg75869
2008-11-14 11:29:20hayposetfiles: + idle_coding_dialog_cancel.patch
keywords: + patch
messages: + msg75858
2008-11-14 11:18:28hayposetfiles: + idle_remove_coding_dialog.py
messages: + msg75857
2008-11-14 11:17:57hayposetfiles: + idle_dialog.png
nosy: + haypo
messages: + msg75856
2008-11-14 11:03:13andrecreate