This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Console UnicodeDecodeError s once more
Type: Stage:
Components: Unicode, Windows Versions: Python 3.0, Python 2.5
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, techtonik
Priority: normal Keywords:

Created on 2008-04-11 08:47 by techtonik, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (2)
msg65342 - (view) Author: anatoly techtonik (techtonik) Date: 2008-04-11 08:47
Python debugging under console is a PITA, because it has a bad habit to
fail with UnicodeEncodeError in case of unknown encoding in output. It
quickly turns into a headache when inspecting methods like in the
following example running under windows:

>>> import active_directory
>>> users = active_directory.search ("objectCategory='Person'",
"objectClass='User'")
>>> u = users.next()
>>> u = users.next()
>>> u = users.next()
>>> u.dump()
LDAP://CN=LыхъёрэфЁ,CN=Users,DC=dom,DC=com
{
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build\bdist.win32\egg\active_directory.py", line 495, in dump
  File "C:\Python25\lib\codecs.py", line 303, in write
    data, consumed = self.encode(object, self.errors)
  File "C:\Python25\lib\encodings\cp1251.py", line 12, in en
code
    return codecs.charmap_encode(input,errors,encoding_table)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x84 in position 8:
ordinal
not in range(128)

Will this be fixed in Py3k to allow range(255) in case of unknown
encoding? Or will there be a new wrapper function - some
rawhexprint(...) that will temporary change sys.output encoding for the
time arguments are executed and process out of range symbols in output
stream according to its embedded rules (i.e. converting to hex
representaton). Another function can be supplied to write raw bytes to
console as-is. Raw write is preferable as it gives the possibility to
redirect output to a file and inspect it later.

These encoding issues in my POV is the stumbling block that makes
scripting in Python 2.x harder for non-english users and hampers overall
Python adoption. Is this going change in Py3k?
msg80056 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-01-17 22:36
I think this had been addressed by PEP 3138 [1].

[1] http://www.python.org/dev/peps/pep-3138/
History
Date User Action Args
2022-04-11 14:56:33adminsetgithub: 46866
2009-01-17 22:36:59benjamin.petersonsetstatus: open -> closed
resolution: out of date
messages: + msg80056
nosy: + benjamin.peterson
2008-04-11 08:47:37techtonikcreate