if i configure logging into a file with encoding = 'cp1251' and do
logger.debug(u'...') then i get crash with UnicodeError

i suggest reimplementing method FileHandler.emit():
...
if isinstance(msg, unicode):
    stream.write(f % msg)    # it works!
...

The attached test script and output file appear to show logging working
correctly. The script writes a log message including the Cyrillic text
доброе утро (Good morning) to a CP1251-encoded file, test.log. Opening
this file in a Unicode-aware editor (I used BabelPad,
http://www.babelstone.co.uk/Software/BabelPad.html) appears to show the
Cyrillic characters correctly: see the screenshot at

http://img5.imageshack.us/img5/799/cp1251zw2.png

Marking as pending. Can you give more specifics about your problem?

test_log.py:
-----------
#! -*- coding: windows-1251 -*-

import logging

logger = logging.getLogger('test_log')
logger.addHandler(logging.FileHandler('test.log', encoding='cp1251'))
logger.setLevel(logging.DEBUG)

logger.debug(u'Привет')    # russian Hello

exception:
---------
Traceback (most recent call last):
  File "e:\bin\python\0\lib\logging\__init__.py", line 765, in emit
    self.stream.write(fs % msg.encode("UTF-8"))
  File "e:\bin\python\0\lib\codecs.py", line 686, in write
    return self.writer.write(data)
  File "e:\bin\python\0\lib\codecs.py", line 351, in write
    data, consumed = self.encode(object, self.errors)
  File "e:\bin\python\0\lib\encodings\cp1251.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_table)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0:
ordinal not in range(128)

Sorry, misread the original issue and tested with Python 2.5 rather than
2.6. This is a regression; fix and additional test case checked into
trunk and release26-maint.

(python 2.6.2, WinXP)
logging to console stopped working. Here is a workaround:

logging/__init__.py:
class StreamHandler(Handler):
    ...
    def emit(self, record):
    ...
                    if (isinstance(msg, unicode) or
                        getattr(stream, 'encoding', None) is None):
-                         stream.write(fs % msg)
+                         if stream == sys.stdout:
+                             print(msg)
+                         else:
+                             stream.write(fs % msg)

Can you retry with setting the "encoding" attribute of the file to
"cp1251"? That should work and that should be the appropriate method to
avoid the problem.

test_logging.py in the Python distribution has a test which exercises
Unicode functionality using cp1251, does that test work in your environment?

>
> Can you retry with setting the "encoding" attribute of the file to
> "cp1251"? That should work and that should be the appropriate method to
> avoid the problem.
>
> test_logging.py in the Python distribution has a test which exercises
> Unicode functionality using cp1251, does that test work in your
> environment?

logging to file is OK (since 2.6.2), logging to console __was__ OK (2.6.1),
not now (2.6.2)

shamil

Trunk and release26-maint were recently changed (r71657, r71658) to use
the following logic, which differs from the code snippet you posted.

                    if (isinstance(msg, unicode) and
                        getattr(stream, 'encoding', None)):
                        stream.write(fs.decode(stream.encoding) % msg)
                    else:
                        stream.write(fs % msg)

If the stream is stderr and you are passing a unicode msg, the else
branch will not be taken; as long as the stream has an encoding
attribute, it should output correctly.

The change was made when another, similar issue was posted by another
user (issue #5768).

Can you confirm what happens with the current code as it is in
release26-maint? спасибо!

>
> Trunk and release26-maint were recently changed (r71657, r71658) to use
> the following logic, which differs from the code snippet you posted.
>
>                    if (isinstance(msg, unicode) and
>                        getattr(stream, 'encoding', None)):
>                        stream.write(fs.decode(stream.encoding) % msg)
>                     else:
>                        stream.write(fs % msg)
>
> If the stream is stderr and you are passing a unicode msg, the else
> branch will not be taken; as long as the stream has an encoding
> attribute, it should output correctly.
>
> The change was made when another, similar issue was posted by another
> user (issue #5768).
>
> Can you confirm what happens with the current code as it is in
> release26-maint? спасибо!

it still doesn't work for console (but OK for files).

the following works in both cases:
if (isinstance(msg, unicode) and
    getattr(stream, 'encoding', None) and
    (stream == sys.stdout or stream == sys.stderr)):
    stream.write(fs % msg.encode(stream.encoding))
else:
     stream.write(fs % msg)

i think it's all about the difference betwin print(msg) and
sys.stdout.write('%s\n' % msg)

shamil

> i think it's all about the difference betwin print(msg) and
> sys.stdout.write('%s\n' % msg)

Yes. Actually, it's about what streams will accept. For example, a
stream opened with codecs.open(encoding='cp1251') will accept a Unicode
string and encode it before output. Likewise, a StringIO wrapped inside
a writer obtained from the codecs module.

However, stdout and stderr are of type file and seem to behave
differently - they behave as expected if I encode the value written to
them, but fail if I write a Unicode string to them.

I don't think special-casing for sys.stdout and sys.stderr in the
logging code is the correct approach. Better to put a conditional
encoding step when it's needed - now I just need to figure out all the
cases when it's needed ;-)

Fix checked into trunk and release26-maint.