Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Audit does not work with non-ASCII data on non-UTF-8 locale #85266

Closed
serhiy-storchaka opened this issue Jun 23, 2020 · 10 comments
Closed

Audit does not work with non-ASCII data on non-UTF-8 locale #85266

serhiy-storchaka opened this issue Jun 23, 2020 · 10 comments
Labels
3.8 only security fixes 3.9 only security fixes 3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

BPO 41094
Nosy @vstinner, @serhiy-storchaka, @zooba, @miss-islington
PRs
  • bpo-41094: Fix decoding errors with audit when open files. #21095
  • [3.9] bpo-41094: Fix decoding errors with audit when open files. (GH-21095) #21107
  • [3.8] bpo-41094: Fix decoding errors with audit when open files. (GH-21095) #21108
  • bpo-41094: Additional fix for PYTHONSTARTUP. #21119
  • [3.9] bpo-41094: Fix decoding errors with audit when open files. (GH-21095) #21120
  • [3.8] bpo-41094: Fix decoding errors with audit when open files. (GH-21095) #21121
  • [3.9] bpo-41094: Additional fix for PYTHONSTARTUP. (GH-21119) #21124
  • [3.8] bpo-41094: Additional fix for PYTHONSTARTUP. (GH-21119) #21125
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-06-24.17:06:51.203>
created_at = <Date 2020-06-23.20:25:16.306>
labels = ['interpreter-core', 'type-bug', '3.8', '3.9', '3.10']
title = 'Audit does not work with non-ASCII data on non-UTF-8 locale'
updated_at = <Date 2020-06-24.17:16:06.330>
user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2020-06-24.17:16:06.330>
actor = 'miss-islington'
assignee = 'none'
closed = True
closed_date = <Date 2020-06-24.17:06:51.203>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2020-06-23.20:25:16.306>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 41094
keywords = ['patch']
message_count = 10.0
messages = ['372205', '372217', '372232', '372234', '372235', '372259', '372260', '372261', '372267', '372268']
nosy_count = 4.0
nosy_names = ['vstinner', 'serhiy.storchaka', 'steve.dower', 'miss-islington']
pr_nums = ['21095', '21107', '21108', '21119', '21120', '21121', '21124', '21125']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41094'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']
    @serhiy-storchaka
    Copy link
    Member Author

    There are issues with using PySys_Audit() with non-ASCII data on non-UTF-8 locale.

    One example is with PYTHONSTARTUP. In pymain_run_startup() in Modules/main.c the value of the PYTHONSTARTUP environment variable is passed to PySys_Audit() as UTF-8 encoded data. If it contains non-ASCII characters and the locale encoding is different from UTF-8, it fails.

    There are similar bugs in _Py_fopen() and _Py_fopen_obj().

    @serhiy-storchaka serhiy-storchaka added 3.8 only security fixes 3.9 only security fixes 3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Jun 23, 2020
    @zooba
    Copy link
    Member

    zooba commented Jun 23, 2020

    SGTM. Thanks for the PR!

    @serhiy-storchaka
    Copy link
    Member Author

    New changeset 6c6810d by Serhiy Storchaka in branch 'master':
    bpo-41094: Fix decoding errors with audit when open files. (GH-21095)
    6c6810d

    @serhiy-storchaka
    Copy link
    Member Author

    Oh, I made a mistake in tests (they do not test with non-ASCII data), but fixed tests fail on Windows.

    @serhiy-storchaka
    Copy link
    Member Author

    The problem is that they pass successfully on my local Windows, but fail on CI.

    @miss-islington
    Copy link
    Contributor

    New changeset 7329c8c by Miss Islington (bot) in branch '3.9':
    bpo-41094: Fix decoding errors with audit when open files. (GH-21095)
    7329c8c

    @miss-islington
    Copy link
    Contributor

    New changeset 1813d31 by Miss Islington (bot) in branch '3.8':
    bpo-41094: Fix decoding errors with audit when open files. (GH-21095)
    1813d31

    @serhiy-storchaka
    Copy link
    Member Author

    New changeset a7dc714 by Serhiy Storchaka in branch 'master':
    bpo-41094: Additional fix for PYTHONSTARTUP. (GH-21119)
    a7dc714

    @miss-islington
    Copy link
    Contributor

    New changeset 1dda40c by Miss Islington (bot) in branch '3.8':
    bpo-41094: Additional fix for PYTHONSTARTUP. (GH-21119)
    1dda40c

    @miss-islington
    Copy link
    Contributor

    New changeset 29aa808 by Miss Islington (bot) in branch '3.9':
    bpo-41094: Additional fix for PYTHONSTARTUP. (GH-21119)
    29aa808

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.8 only security fixes 3.9 only security fixes 3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @zooba @serhiy-storchaka @miss-islington