Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

localeconv() does not encode returned strings #46279

Closed
pitrou opened this issue Feb 1, 2008 · 6 comments
Closed

localeconv() does not encode returned strings #46279

pitrou opened this issue Feb 1, 2008 · 6 comments
Labels
extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error

Comments

@pitrou
Copy link
Member

pitrou commented Feb 1, 2008

BPO 1995
Nosy @loewis, @pitrou
Files
  • pylocaleconv.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2008-03-08.10:43:45.174>
    created_at = <Date 2008-02-01.18:13:47.482>
    labels = ['extension-modules', 'type-bug']
    title = 'localeconv() does not encode returned strings'
    updated_at = <Date 2008-03-08.10:43:45.173>
    user = 'https://github.com/pitrou'

    bugs.python.org fields:

    activity = <Date 2008-03-08.10:43:45.173>
    actor = 'loewis'
    assignee = 'none'
    closed = True
    closed_date = <Date 2008-03-08.10:43:45.174>
    closer = 'loewis'
    components = ['Extension Modules']
    creation = <Date 2008-02-01.18:13:47.482>
    creator = 'pitrou'
    dependencies = []
    files = ['9343']
    hgrepos = []
    issue_num = 1995
    keywords = []
    message_count = 6.0
    messages = ['61970', '61971', '61973', '61977', '61987', '63395']
    nosy_count = 2.0
    nosy_names = ['loewis', 'pitrou']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = None
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue1995'
    versions = ['Python 3.0']

    @pitrou
    Copy link
    Member Author

    pitrou commented Feb 1, 2008

    Some values in the dict returned by localeconv() may be non-ASCII
    strings, yet they are not decoded according to the locale's character
    set. This can be observed when the currency symbol is the euro sign:

    >>> import locale
    >>> locale.setlocale(locale.LC_MONETARY, 'fr_FR.UTF-8')
    'fr_FR.UTF-8'
    >>> locale.localeconv()['currency_symbol']
    '\xe2\x82\xac'
    >>> locale.setlocale(locale.LC_MONETARY, 'fr_FR.ISO8859-15')
    'fr_FR.ISO8859-15'
    >>> locale.localeconv()['currency_symbol']
    '\xa4'

    localeconv() is defined in the _locale module, which has no knowledge of
    the current encoding - but the locale module does. So we could redefine
    localeconv() in locale as a wrapper, to do the proper encoding dance.

    @pitrou pitrou added extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error labels Feb 1, 2008
    @pitrou
    Copy link
    Member Author

    pitrou commented Feb 1, 2008

    Here is a patch fixing the problem. Note however that it will make
    localeconv() quite slower. Perhaps _locale.localeconv should grow an
    encoding parameter instead.

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Feb 1, 2008

    The locale module is completely broken; don't try to work around that
    breakage. One option would be to remove it entirely.

    @pitrou
    Copy link
    Member Author

    pitrou commented Feb 1, 2008

    Perhaps it is broken - it does look rather fragile - but are there any
    plans to design a replacement?

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Feb 2, 2008

    Perhaps it is broken - it does look rather fragile - but are there any
    plans to design a replacement?

    If people could contribute an ICU wrapper - that would be nice. However,
    it's unlikely to happen. So I'll rather rewrite _locale to use wchar_t
    functions, and give up on systems where these are not available, or
    where wchar_t is not Unicode (not sure how to detect the latter case).

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Mar 8, 2008

    I found now a way to fix this, by relying on wchar_t functions. It's
    fixed in r61306

    @loewis loewis mannequin closed this as completed Mar 8, 2008
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant