Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FreeBSD: Fatal Python error: get_locale_encoding: failed to get the locale encoding: nl_langinfo(CODESET) failed #78725

Closed
vstinner opened this issue Aug 29, 2018 · 8 comments
Closed

FreeBSD: Fatal Python error: get_locale_encoding: failed to get the locale encoding: nl_langinfo(CODESET) failed #78725

vstinner opened this issue Aug 29, 2018 · 8 comments
Labels
3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@vstinner
Copy link
Member

BPO 34544
Nosy @vstinner
PRs
  • bpo-34544: _Py_CoerceLegacyLocale() restores LC_CTYPE on fail #9044
  • bpo-34544: pymain_read_conf() don't change LC_ALL #9045
  • [3.7] bpo-34544: _Py_CoerceLegacyLocale() restores LC_CTYPE on fail (GH-9044) #9046
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-09-03.20:17:51.296>
created_at = <Date 2018-08-29.23:37:46.027>
labels = ['interpreter-core', '3.8']
title = 'FreeBSD: Fatal Python error: get_locale_encoding: failed to get the locale encoding: nl_langinfo(CODESET) failed'
updated_at = <Date 2018-09-03.20:17:51.294>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2018-09-03.20:17:51.294>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-09-03.20:17:51.296>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2018-08-29.23:37:46.027>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 34544
keywords = ['patch']
message_count = 8.0
messages = ['324361', '324378', '324512', '324516', '324522', '324527', '324533', '324534']
nosy_count = 1.0
nosy_names = ['vstinner']
pr_nums = ['9044', '9045', '9046']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue34544'
versions = ['Python 3.8']
    @vstinner
    Copy link
    Member Author

    It seems like my recent work on Python initialization broke Python on FreeBSD. Many test_c_locale_coerce tests now fail with:

    "Fatal Python error: get_locale_encoding: failed to get the locale encoding: nl_langinfo(CODESET) failed"

    Example of failed buildbot build:

    https://buildbot.python.org/all/#/builders/87/builds/1340

    It seems like this buildbot started to fail at this build:

    https://buildbot.python.org/all/#/builders/87/builds/1332

    This build has been triggered by this change:

    commit 177d921
    Author: Victor Stinner <vstinner@redhat.com>
    Date: Wed Aug 29 11:25:15 2018 +0200

    bpo-34485, Windows: LC_CTYPE set to user preference (GH-8988)

On Windows, the LC_CTYPE is now set to the user preferred locale at
startup: _Py_SetLocaleFromEnv(LC_CTYPE) is now called during the
Python initialization. Previously, the LC_CTYPE locale was "C" at
startup, but changed when calling setlocale(LC_CTYPE, "") or
setlocale(LC_ALL, "").

pymain_read_conf() now also calls _Py_SetLocaleFromEnv(LC_CTYPE) to
behave as _Py_InitializeCore(). Moreover, it doesn't save/restore the
LC_ALL anymore.

On Windows, standard streams like sys.stdout now always use
surrogateescape error handler by default (ignore the locale).

    test_c_locale_coerce tests invalid locales like LANG="invalid.ascii". It seems like Python supported such configuration before my changes.

    My recent work:

    @vstinner vstinner added 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Aug 29, 2018
    @vstinner
    Copy link
    Member Author

    I cannot reproduce the issue on my FreeBSD 11.1 VM. I cannot reproduce the issue on Koobs's "CURRENT-amd64%" buildbot neither :-(

    Example:

    CURRENT-amd64% env -i LC_ALL=invalid LC_CTYPE=invalid LANG=invalid PYTHONUTF8=0 PYTHONCOERCECLOCALE=0 ./python -X utf8=0 -c pass

    (no error)

    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 3, 2018

    New changeset f01b2a1 by Victor Stinner in branch 'master':
    bpo-34544: Fix setlocale() in pymain_read_conf() (GH-9041)
    f01b2a1

    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 3, 2018

    My fix didn't work :-(

    https://buildbot.python.org/all/#/builders/87/builds/1355

    ...
    Fatal Python error: get_locale_encoding: failed to get the locale encoding: nl_langinfo(CODESET) failed
    ...

    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 3, 2018

    New changeset 8ea0911 by Victor Stinner in branch 'master':
    _Py_CoerceLegacyLocale() restores LC_CTYPE on fail (GH-9044)
    8ea0911

    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 3, 2018

    New changeset 73b00be by Victor Stinner in branch 'master':
    bpo-34544: pymain_read_conf() don't change LC_ALL (GH-9045)
    73b00be

    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 3, 2018

    New changeset 84b0129 by Victor Stinner in branch '3.7':
    _Py_CoerceLegacyLocale() restores LC_CTYPE on fail (GH-9044) (GH-9046)
    84b0129

    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 3, 2018

    test_c_locale_coerce pass again on AMD64 FreeBSD 10.x Shared 3.x, I close the issue.

    @vstinner vstinner closed this as completed Sep 3, 2018
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @vstinner