On Windows, mimetypes initialization reads the list of MIME types from the Windows registry. It assumes that all characters are Latin-1 encoded, and fails when it's not the case, with the following exception:

Traceback (most recent call last):
  File "mttest.py", line 3, in <module>
    mimetypes.init()
  File "c:\Python27\lib\mimetypes.py", line 355, in init
    db.read_windows_registry()
  File "c:\Python27\lib\mimetypes.py", line 260, in read_windows_registry
    for ctype in enum_types(mimedb):
  File "c:\Python27\lib\mimetypes.py", line 250, in enum_types
    ctype = ctype.encode(default_encoding) # omit in 3.x!
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)

This can be reproduced, for example, on a Russian Windows XP installation which has QuickTime installed (QuickTime creates the non-Latin entries in the registry). The following line causes the exception to happen:

import mimetypes; mimetypes.init()

I'm guessing this problem doesn't occur in 3.x?  If so, the quick fix would be to have the registry code catch UnicodeError instead of UnicodeEncodeError.  That may be the correct fix anyway.

The "fun" part of this bug is going to be creating a unit test for it.

The problem doesn't happen on Python 3.1.2 because it doesn't have the code in mimetypes that accesses the Windows registry. Haven't tried the 3.2 alphas yet.

Patch (suggested fix and unittest) attached.

And by the way I've verified that the problem doesn't happen in py3k trunk.

And just for clarity: py3k trunk does contain the _winreg code path.

python 3.1.2 mimetypes initialization also fails in redhat linux:


>>> import http.server
Traceback (most recent call last):
  File "/home/public/i386-redhat-linux-gnu/python/lib/python3.1/http/server.py", line 588, in <module>
    class SimpleHTTPRequestHandler(BaseHTTPRequestHandler):
  File "/home/public/i386-redhat-linux-gnu/python/lib/python3.1/http/server.py", line 764, in SimpleHTTPRequestHandler
    mimetypes.init() # try to read system mime.types
  File "/home/public/i386-redhat-linux-gnu/python/lib/python3.1/mimetypes.py", line 305, in init
    db.readfp(open(file))
  File "/home/public/i386-redhat-linux-gnu/python/lib/python3.1/mimetypes.py", line 209, in readfp
    line = fp.readline()
  File "/home/public/i386-redhat-linux-gnu/bin/../python/lib/python3.1/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 3921: ordinal not in range(128)

Here is a better patch.

UnicodeDecodeException is thrown because 'ctype' is already a string, 
so it is first implicitly decoded by default encoder (which is 'ascii') and then reencoded back. I see no reason in all these actions, so I simply removed them. I think Antoine Pitrou (who is the author of these lines) can shed some light on this, but I guess it's just a copy-paste of the code below.