Author jmfauth
Recipients amaury.forgeotdarc, benjamin.peterson, brett.cannon, jmfauth, sjmachin, vstinner
Date 2009-03-25.14:56:47
SpamBayes Score 1.54726e-11
Marked as misclassified No
Message-id <1237993010.71.0.488597004993.issue4626@psf.upfronthosting.co.za>
In-reply-to
Content
> Victor

Yes, I could, but I think this is not an IDLE issue, I'm just
using IDLE to illustrate the problem. I got the same when I'm
working from within an editor or with my interactive interpreter
I wrote for the fun.

Code in the editor:

# -*- coding: cp1252 -*-
# Python 3.0.1, winxp sp2

from code import InteractiveInterpreter

ii = InteractiveInterpreter()
source = b'print(999)'
ii.runsource(source)

Output:

>c:\python30\pythonw -u "uuu.py"
Traceback (most recent call last):
  File "uuu.py", line 8, in <module>
    ii.runsource(source)
  File "c:\python30\lib\code.py", line 63, in runsource
    code = self.compile(source, filename, symbol)
  File "c:\python30\lib\codeop.py", line 168, in __call__
    return _maybe_compile(self.compiler, source, filename, symbol)
  File "c:\python30\lib\codeop.py", line 70, in _maybe_compile
    for line in source.split("\n"):
TypeError: Type str doesn't support the buffer API
>Exit code: 1

My interactive interpreter

>>> ---
from code import InteractiveInterpreter
>>> ---
ii = InteractiveInterpreter()
>>> ---
source = b'print(999)'
>>> ---
ii.runsource(source)
Traceback (most recent call last):
  File "<smid last command>", line 1, in <module>
  File "c:\Python30\lib\code.py", line 63, in runsource
    code = self.compile(source, filename, symbol)
  File "c:\Python30\lib\codeop.py", line 168, in __call__
    return _maybe_compile(self.compiler, source, filename, symbol)
  File "c:\Python30\lib\codeop.py", line 70, in _maybe_compile
    for line in source.split("\n"):
TypeError: Type str doesn't support the buffer API
>>> ---

=======================

I realised and missed the fact the str() function is now accepting
an encoding argument (very good). With it, the above code works.

Code in the editor:

# -*- coding: cp1252 -*-
# Python 3.0.1, winxp sp2

from code import InteractiveInterpreter

ii = InteractiveInterpreter()
source = b'print(999)'
source = str(source, 'cp1252') #<<<<<<<<<<
ii.runsource(source)

Output: (ok)

>c:\python30\pythonw -u "uuu.py"
999
>Exit code: 0

=======================

In a few words, my empirical understanding of the story.

1) Things are in good shape, but Python, itsself and its
components (compile, interactiveinterpreter module, runsource(),
exec(), ...) are lacking in consistency, the all accept 
miscellanous arguments type or they are not strict enough in 
their arguments acceptance. When they accept a str, which 
encoding is accepted?


2) Python 3 is now using unicode as str. Of course, this is welcome,
but the caveat is that there are now two encodings in use. Code source
defaults to utf-8, but the str in code defaults to ucs-2/4 (eg. in
IDLE).

3) Maybe a solution is to have an optional encoding argument as we now 
have everywhere eg. str(), open(), which defaults to one encoding.

compile(source, filename, encodings='utf-8', ...)
(Problem: BOM, coding cookies?).

I suspect the miscellaneous discussions one finds from people attempting
to write a "correct" execfile() for Python 3 are coming from this.

Regards.
History
Date User Action Args
2009-03-25 14:56:51jmfauthsetrecipients: + jmfauth, brett.cannon, sjmachin, amaury.forgeotdarc, vstinner, benjamin.peterson
2009-03-25 14:56:50jmfauthsetmessageid: <1237993010.71.0.488597004993.issue4626@psf.upfronthosting.co.za>
2009-03-25 14:56:49jmfauthlinkissue4626 messages
2009-03-25 14:56:47jmfauthcreate