classification
Title: IDLE ignores -*- coding -*- with -r option
Type: behavior Stage:
Components: IDLE Versions: Python 3.2, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: haypo, ledave123, python-dev
Priority: normal Keywords:

Created on 2011-07-25 09:52 by ledave123, last changed 2011-09-01 23:09 by haypo. This issue is now closed.

Messages (6)
msg141081 - (view) Author: (ledave123) Date: 2011-07-25 09:52
I'm on Windows with cp1252 as the default encoding.
When I use -*- coding: c1252 -*- I get no problems.
When I use -*- coding: utf-8 -*- IDLE -r still opens the file with cp1252 encoding.
Python.exe opens the file with utf-8 correctly.

I think the problem is in Python32\Lib\idlelib\PyShell.py line 585:
In class ModifiedInterpreter:

    def execfile(self, filename, source=None):
        "Execute an existing file"
        if source is None:
            source = open(filename, "r").read() # this is the bug IMHO
msg141285 - (view) Author: (ledave123) Date: 2011-07-28 08:11
The problem can be fixed with tokenize :
I'm sorry I never submitted a path and I have no access to the source tree from here, if someone cares to do it, do not hesitate.

    def execfile(self, filename, source=None):
        "Execute an existing file"
        if source is None:
            import tokenize
            source = tokenize.open(filename).read()
msg141287 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-07-28 08:28
Yes, tokenize.open() should fix this issue, but you should close the file after using it. Use for example "with tokenize.open(): ...".

Can you write a patch? You can download the source code using Mercurial or download it manually from http://hg.python.org/cpython/file/tip

See also our http://docs.python.org/devguide/
msg143273 - (view) Author: (ledave123) Date: 2011-08-31 18:20
Here is the patch:

diff -r e8da570d29a8 Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py    Wed Jul 27 21:28:23 2011 +0200
+++ b/Lib/idlelib/PyShell.py    Wed Aug 31 20:16:38 2011 +0200
@@ -582,7 +582,9 @@
     def execfile(self, filename, source=None):
         "Execute an existing file"
         if source is None:
-            source = open(filename, "r").read()
+            import tokenize
+            with tokenize.open(filename) as filein:
+                source = filein.read()
         try:
             code = compile(source, filename, "exec")
         except (OverflowError, SyntaxError):

Sorry for taking such a long time, I was on holidays.
msg143362 - (view) Author: Roundup Robot (python-dev) Date: 2011-09-01 23:05
New changeset fe7777f1ed14 by Victor Stinner in branch '3.2':
Issue #12636: IDLE reads the coding cookie when executing a Python script.
http://hg.python.org/cpython/rev/fe7777f1ed14

New changeset a8748022504f by Victor Stinner in branch 'default':
Merge 3.2: Issue #12636: IDLE reads the coding cookie when executing a Python script.
http://hg.python.org/cpython/rev/a8748022504f
msg143363 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-09-01 23:06
Fixed, thanks for the report.

Python 2.7 is not affected by this bug.
History
Date User Action Args
2011-09-01 23:09:04hayposetstatus: open -> closed
resolution: works for me -> fixed
2011-09-01 23:06:49hayposetmessages: + msg143363
versions: + Python 3.3
2011-09-01 23:05:22python-devsetnosy: + python-dev
messages: + msg143362
2011-08-31 18:20:46ledave123setresolution: works for me
messages: + msg143273
2011-07-28 08:28:51hayposetmessages: + msg141287
2011-07-28 08:11:42ledave123setmessages: + msg141285
2011-07-27 19:26:00hayposetnosy: + haypo
2011-07-25 09:52:55ledave123create