Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mimetypes should read the rule file using UTF-8, not the locale encoding #57234

Closed
vstinner opened this issue Sep 20, 2011 · 5 comments
Closed

mimetypes should read the rule file using UTF-8, not the locale encoding #57234

vstinner opened this issue Sep 20, 2011 · 5 comments
Labels
stdlib Python modules in the Lib dir topic-unicode type-bug An unexpected behavior, bug, or error

Comments

@vstinner
Copy link
Member

BPO 13025
Nosy @terryjreedy, @vstinner, @merwok, @sandrotosi, @serhiy-storchaka
Files
  • mimetypes_encoding.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2011-10-14.01:04:46.668>
created_at = <Date 2011-09-20.23:07:07.202>
labels = ['type-bug', 'library', 'expert-unicode']
title = 'mimetypes should read the rule file using UTF-8, not the locale encoding'
updated_at = <Date 2020-06-20.09:11:44.600>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2020-06-20.09:11:44.600>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2011-10-14.01:04:46.668>
closer = 'vstinner'
components = ['Library (Lib)', 'Unicode']
creation = <Date 2011-09-20.23:07:07.202>
creator = 'vstinner'
dependencies = []
files = ['23219']
hgrepos = []
issue_num = 13025
keywords = ['patch']
message_count = 5.0
messages = ['144357', '144455', '145493', '145494', '371926']
nosy_count = 6.0
nosy_names = ['terry.reedy', 'vstinner', 'eric.araujo', 'sandro.tosi', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'test needed'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue13025'
versions = ['Python 3.2', 'Python 3.3']
    @vstinner
    Copy link
    Member Author

    On Debian and Ubuntu, /etc/mime.types file is pure ASCII, but on Fedora 15 it contains a non-ASCII character, ³ (U+00B3), in the line:
    "application/vnd.geocube+xml g3 g³"

    And the file is encoded in UTF-8.

    That's why Python should read this file from UTF-8 instead of the locale encoding, because the locale encoding can be ASCII. Attached patch implements this idead.

    I think that it is a bug and so it should also be fixed in Python 3.2.

    (Python 2.7 reads the file in binary mode, it doesn't care of the encoding.)

    @vstinner vstinner added stdlib Python modules in the Lib dir topic-unicode labels Sep 20, 2011
    @merwok
    Copy link
    Member

    merwok commented Sep 23, 2011

    +1. I’ve finally understood that open using the locale is Evil™. Please use the file from Fedora in a test.

    @ezio-melotti ezio-melotti added the type-bug An unexpected behavior, bug, or error label Sep 23, 2011
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Oct 14, 2011

    New changeset 8d8ab3e04363 by Victor Stinner in branch '3.2':
    Issue bpo-13025: mimetypes is now reading MIME types using the UTF-8 encoding,
    http://hg.python.org/cpython/rev/8d8ab3e04363

    New changeset 2c223d686feb by Victor Stinner in branch 'default':
    (Merge 3.2) Issue bpo-13025: mimetypes is now reading MIME types using the UTF-8
    http://hg.python.org/cpython/rev/2c223d686feb

    @vstinner
    Copy link
    Member Author

    Please use the file from Fedora in a test.

    Done.

    @serhiy-storchaka
    Copy link
    Member

    However read_mime_types() still uses the locale encoding. See bpo-41048.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    stdlib Python modules in the Lib dir topic-unicode type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @vstinner @merwok @serhiy-storchaka @ezio-melotti