classification
Title: Fatal LookupError: unknown encoding: cp0 on Windows embedded startup.
Type: crash Stage:
Components: Interpreter Core, Windows Versions: Python 3.1
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, grahamd, pitrou (3)
Priority: Keywords

Created on 2009-07-17 10:21 by grahamd, last changed 2009-08-05 12:08 by grahamd.

Messages (7)
msg90616 - (view) Author: Graham Dumpleton (grahamd) Date: 2009-07-17 10:21
When using Python 3.1 for Apache/mod_wsgi (3.0c4) on Windows, Apache will 
crash on startup because Python is forcing the process to exit with:

Fatal Python error: Py_Initialize: can't initialize sys standard streams
LookupError: unknown encoding: cp0

I first mentioned this on issue6393, but have now created it as a separate 
issue as appears to be distinct from the issue on MacOS X, athough possibly 
related.

In the Windows case there is actually an encoding, that of 'cp0' where as on 
MacOS X, the encoding name was empty.

The same mod_wsgi code works fine under Python 3.1 on MacOS X.
msg90618 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) Date: 2009-07-17 11:20
- Apache is not a Console application, so the Windows GetConsoleCP()
function returns zero (and os.device_encoding(1) returns 'cp0').
- pythonw.exe has no console either; but in pythonrun.c, the test
(fileno(stdin) < 0) is true, and the standard streams are all set to None.
- It is probable that Apache has redefined stdin & co, so the previous
test does not work there.

As a workaround, I suggest to set the environment variable
PYTHONIOENCODING before starting Apache, or before the call to
Py_Initialize.
msg90620 - (view) Author: Graham Dumpleton (grahamd) Date: 2009-07-17 11:32
Yes, Apache remaps stdout and stderr to the Apache error log to still 
capture anything that errant modules don't log via the Apache error log 
functions. In mod_wsgi it replaces sys.stdout and sys.stderr with special 
file like objects that redirect via Apache error logging functions. This 
though obviously happens after Python first initialises sys.stdout and 
sys.stderr.

What would be an appropriate value to set PYTHONIOENCODING to on Windows 
as a workaround?
msg90640 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) Date: 2009-07-17 20:50
On a Western Windows, I suggest
    PYTHONIOENCODING=cp1252:backslashreplace

But 
    PYTHONIOENCODING=mbcs
is also OK, except that characters outside the Windows code page will be 
replaced with '?'
msg90813 - (view) Author: Graham Dumpleton (grahamd) Date: 2009-07-22 12:15
The workaround of using:

#if defined(WIN32) && PY_MAJOR_VERSION >= 3
        _wputenv(L"PYTHONIOENCODING=cp1252:backslashreplace");
#endif

        Py_Initialize();

gets around the crash on startup.

I haven't done sufficient testing to know if this may introduce any other 
problems given that one is overriding default I/O encoding for whole 
process.
msg91311 - (view) Author: Antoine Pitrou (pitrou) Date: 2009-08-05 11:01
Graham, is the workaround ok for you or do you think this is something
Python itself should handle?
msg91317 - (view) Author: Graham Dumpleton (grahamd) Date: 2009-08-05 12:08
Python should be as robust as possible and thus should be fixed, but I am 
happy with using the workaround so if this is more a question of what 
priority to mark this, I wouldn't see it as being urgent.
History
Date User Action Args
2009-08-05 12:08:58grahamdsetmessages: + msg91317
2009-08-05 11:01:37pitrousetnosy: + pitrou
messages: + msg91311
2009-07-22 12:15:32grahamdsetmessages: + msg90813
2009-07-17 20:50:37amaury.forgeotdarcsetmessages: + msg90640
2009-07-17 11:32:26grahamdsetmessages: + msg90620
2009-07-17 11:20:10amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg90618
2009-07-17 10:21:43grahamdcreate