Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

On FreeBSD, Python 3 doesn't support support the POSIX locale #78708

Closed
vstinner opened this issue Aug 28, 2018 · 7 comments
Closed

On FreeBSD, Python 3 doesn't support support the POSIX locale #78708

vstinner opened this issue Aug 28, 2018 · 7 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@vstinner
Copy link
Member

BPO 34527
Nosy @ncoghlan, @vstinner
PRs
  • bpo-34527: POSIX locale enables the UTF-8 Mode #8972
  • [3.7] bpo-34527: POSIX locale enables the UTF-8 Mode (GH-8972) #8974
  • [3.6] bpo-34527: Fix FreeBSD with POSIX locale #8975
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-08-28.13:06:41.173>
created_at = <Date 2018-08-28.09:35:24.881>
labels = ['interpreter-core', '3.7', '3.8']
title = "On FreeBSD, Python 3 doesn't support support the POSIX locale"
updated_at = <Date 2018-08-29.09:02:23.922>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2018-08-29.09:02:23.922>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-08-28.13:06:41.173>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2018-08-28.09:35:24.881>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 34527
keywords = ['patch']
message_count = 7.0
messages = ['324230', '324234', '324240', '324241', '324247', '324250', '324310']
nosy_count = 2.0
nosy_names = ['ncoghlan', 'vstinner']
pr_nums = ['8972', '8974', '8975']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue34527'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']
    @vstinner
    Copy link
    Member Author

    vstinner@freebsd$ env -i LC_ALL=POSIX ./python -m test -v test_cmd_line_script -m test_non_ascii
    == CPython 3.8.0a0 (heads/master:096329f0b2, Aug 28 2018, 09:30:55) [Clang 4.0.0 (tags/RELEASE_400/final 297347)]
    == FreeBSD-11.1-RELEASE-amd64-64bit-ELF little-endian
    == cwd: /usr/home/vstinner/prog/python/master/build/test_python_13073
    == CPU count: 2
    == encodings: locale=US-ASCII, FS=ascii
    Run tests sequentially
    0:00:00 load avg: 0.48 [1/1] test_cmd_line_script
    test_non_ascii (test.test_cmd_line_script.CmdLineTest) ... FAIL

    ======================================================================
    FAIL: test_non_ascii (test.test_cmd_line_script.CmdLineTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/usr/home/vstinner/prog/python/master/Lib/test/test_cmd_line_script.py", line 512, in test_non_ascii
    'stdout=%r stderr=%r' % (stdout, stderr))
AssertionError: "'./@test_13073_tmp\\udce7w\\udcf0.py'" != "'<filename encoding error>'"
- './@test_13073_tmp\udce7w\udcf0.py'
+ '<filename encoding error>'
 : stdout=b"'<filename encoding error>'\n" stderr=b"Failed checking if argv[0] is an import path entry\nUnicodeEncodeError: 'ascii' codec can't encode character '\\xe7' in position 17: ordinal not in range(128)"

    Ran 1 test in 0.093s

    FAILED (failures=1)
    test test_cmd_line_script failed
    test_cmd_line_script failed

    == Tests result: FAILURE ==

    1 test failed:
    test_cmd_line_script

    Total duration: 170 ms
    Tests result: FAILURE

    By the way, the UTF-8 Mode is not enabled by the POSIX locale:

    vstinner@freebsd$ env -i LC_ALL=POSIX ./python -c 'import sys; print(sys.flags.utf8_mode)'
    0

    It contradicts the PEP-540 which says the the UTF-8 Mode is enabled by the POSIX locale:

    https://www.python.org/dev/peps/pep-0540/#abstract

    This mode is off by default, but is automatically activated when using the "POSIX" locale.

    @vstinner vstinner added 3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Aug 28, 2018
    @vstinner
    Copy link
    Member Author

    New changeset 5cb2589 by Victor Stinner in branch 'master':
    bpo-34527: POSIX locale enables the UTF-8 Mode (GH-8972)
    5cb2589

    @vstinner
    Copy link
    Member Author

    New changeset 65ef742 by Victor Stinner in branch '3.7':
    bpo-34527: POSIX locale enables the UTF-8 Mode (GH-8972) (GH-8974)
    65ef742

    @vstinner
    Copy link
    Member Author

    Note: Python 2.7 is not affected: it doesn't have "force_ascii" and "LC_ALL=POSIX ./python -m test test_cmd_line_script" pass successfully.

    @vstinner
    Copy link
    Member Author

    New changeset 4ff3887 by Victor Stinner in branch '3.6':
    bpo-34527: Fix FreeBSD with POSIX locale (GH-8975)
    4ff3887

    @vstinner
    Copy link
    Member Author

    I wanted to quickly fix this issue since I'm working on 3 different issues where I got conflicts with these changes, and the other issues were impacted by this bug.

    I sent an email to python-dev about this issue:
    https://mail.python.org/pipermail/python-dev/2018-August/155052.html

    @vstinner
    Copy link
    Member Author

    See also bpo-34485. Example:

    New changeset 0b9ea4b by Victor Stinner in branch '3.7':
    [3.7] bpo-34485: stdout uses surrogateescape on POSIX locale (GH-8986) (GH-8987)
    0b9ea4b

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @vstinner