Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

datetime: Tests for potential crashes due to non-UTF-8-encodable strings #78663

Closed
izbyshev mannequin opened this issue Aug 23, 2018 · 8 comments
Closed

datetime: Tests for potential crashes due to non-UTF-8-encodable strings #78663

izbyshev mannequin opened this issue Aug 23, 2018 · 8 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes extension-modules C modules in the Modules dir type-security A security issue

Comments

@izbyshev
Copy link
Mannequin

izbyshev mannequin commented Aug 23, 2018

BPO 34482
Nosy @abalkin, @vstinner, @taleinat, @serhiy-storchaka, @pganssle, @izbyshev, @miss-islington
PRs
  • bpo-34482: Add tests for proper handling of non-UTF-8-encodable strin… #8878
  • bpo-34454: datetime: Fix crash on PyUnicode_AsUTF8AndSize() failure #8850
  • [3.7] bpo-34482: test datetime classes' handling of non-UTF-8-encodable strings (GH-8878) #10049
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-10-23.07:06:16.562>
created_at = <Date 2018-08-23.17:41:50.810>
labels = ['type-security', 'extension-modules', '3.7', '3.8']
title = 'datetime: Tests for potential crashes due to non-UTF-8-encodable strings'
updated_at = <Date 2018-11-27.16:17:11.783>
user = 'https://github.com/izbyshev'

    bugs.python.org fields:

    activity = <Date 2018-11-27.16:17:11.783>
actor = 'izbyshev'
assignee = 'none'
closed = True
closed_date = <Date 2018-10-23.07:06:16.562>
closer = 'taleinat'
components = ['Extension Modules']
creation = <Date 2018-08-23.17:41:50.810>
creator = 'izbyshev'
dependencies = []
files = []
hgrepos = []
issue_num = 34482
keywords = ['patch']
message_count = 8.0
messages = ['323964', '324201', '324264', '328282', '328288', '328289', '330484', '330535']
nosy_count = 7.0
nosy_names = ['belopolsky', 'vstinner', 'taleinat', 'serhiy.storchaka', 'p-ganssle', 'izbyshev', 'miss-islington']
pr_nums = ['8878', '8850', '10049']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'security'
url = 'https://bugs.python.org/issue34482'
versions = ['Python 3.7', 'Python 3.8']
    @izbyshev
    Copy link
    Mannequin Author

    izbyshev mannequin commented Aug 23, 2018

    This is a follow-up of bpo-34454. 'datetime' extension module attempts to encode input strings into UTF-8 in several places, which requires special care because some valid Python strings can't be represented in UTF-8. It makes sense to add more tests for methods dealing with strings.

    Note that my PR doesn't attempt to deal with bpo-34481. In cases where behavior differs between C and Python datetime impls the tests check only for absence of crashes.

    @izbyshev izbyshev mannequin added 3.7 (EOL) end of life 3.8 only security fixes extension-modules C modules in the Modules dir type-security A security issue labels Aug 23, 2018
    @pganssle
    Copy link
    Member

    Somewhat related: bpo-6697.

    Turns out there are already some tests here for this, specifically for the C version only: https://github.com/python/cpython/blob/master/Lib/test/datetimetester.py#L3328

    @izbyshev
    Copy link
    Mannequin Author

    izbyshev mannequin commented Aug 28, 2018

    Yes, I've referenced the relevant message from bpo-6697 in bpo-34454.

    The specific test you've referenced should be changed after bpo-34481 is fixed -- it highlights inconsistency between C and Python implementations.

    @taleinat
    Copy link
    Contributor

    New changeset 3b0047d by Tal Einat (Alexey Izbyshev) in branch 'master':
    bpo-34482: test datetime classes' handling of non-UTF-8-encodable strings (GH-8878)
    3b0047d

    @miss-islington
    Copy link
    Contributor

    New changeset 313e501 by Miss Islington (bot) in branch '3.7':
    bpo-34482: test datetime classes' handling of non-UTF-8-encodable strings (GH-8878)
    313e501

    @taleinat
    Copy link
    Contributor

    Thanks for the PR, Alexey!

    @vstinner
    Copy link
    Member

    It seems like thie issue introduced a memory leak: bpo-35322 .

    @izbyshev
    Copy link
    Mannequin Author

    izbyshev mannequin commented Nov 27, 2018

    The added test exposed a leak in unicode_encode_locale(). See msg330534.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life 3.8 only security fixes extension-modules C modules in the Modules dir type-security A security issue
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @vstinner @taleinat @pganssle @miss-islington