classification
Title: Inconsistency between string.letters and default encoding.
Type: Stage:
Components: Windows Versions: Python 2.5
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: loewis, ramong
Priority: normal Keywords:

Created on 2008-08-05 00:33 by ramong, last changed 2008-08-05 05:12 by loewis. This issue is now closed.

Messages (2)
msg70725 - (view) Author: Ramon Garcia (ramong) Date: 2008-08-05 00:33
In python on Windows, under Idle, the string.letters includes extended
characters. But the default codec, used when translating from string to
unicode, is still ascii. This behaviour causes crashes with python win32
extensions.

>>> string.letters

'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz\x83\x8a\x8c\x8e\x9a\x9c\x9e\x9f\xaa\xb5\xba\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff'

But still, unless the user customizes the installation,
sys.getdefaultencoding() returns ascii.

The consequence is that after instating a COM object, pywin32 211 issues
this exception:

  File "C:\Python25\Lib\site-packages\win32com\client\build.py", line
297, in MakeFuncMethod
    return self.MakeDispatchFuncMethod(entry, name, bMakeClass)
  File "C:\Python25\Lib\site-packages\win32com\client\build.py", line
318, in MakeDispatchFuncMethod
    s = linePrefix + 'def ' + name + '(self' + BuildCallList(fdesc,
names, defNamedOptArg, defNamedNotOptArg, defUnnamedArg, defOutArg) + '):'
  File "C:\Python25\Lib\site-packages\win32com\client\build.py", line
604, in BuildCallList
    argName = MakePublicAttributeName(argName)
  File "C:\Python25\Lib\site-packages\win32com\client\build.py", line
542, in MakePublicAttributeName
    return filter( lambda char: char in valid_identifier_chars, className)
  File "C:\Python25\Lib\site-packages\win32com\client\build.py", line
542, in <lambda>
    return filter( lambda char: char in valid_identifier_chars, className)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x83 in position 52:
ordinal not in range(128)



The line that causes this exception is from win32com.client.build.

This fragment is enough to reproduce the bug (from build.py in
win32com/client):

valid_identifier_chars = string.letters + string.digits + "_"
...
return filter( lambda char: char in valid_identifier_chars, className)

Try to print the expression in the return statement and set className to
anything you wish in Unicode. It will crash

It is contradictory that the default codec does not allow translation of
characters  0x83, and that string.letters includes it. If one regards
this character as printable, then it should be encoded successfully.
msg70730 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-08-05 05:12
That's a bug in the Win32 extensions. They shouldn't use string.letters,
but string.ascii_letters, in particular when they check for valid
identifier chars.

Closing this report as "won't fix".
History
Date User Action Args
2008-08-05 05:12:47loewissetstatus: open -> closed
resolution: wont fix
messages: + msg70730
nosy: + loewis
2008-08-05 00:33:52ramongcreate