Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PyString_FromFormatV() fails to build empty strings #77998

Closed
Tey mannequin opened this issue Jun 9, 2018 · 6 comments
Closed

PyString_FromFormatV() fails to build empty strings #77998

Tey mannequin opened this issue Jun 9, 2018 · 6 comments
Assignees
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@Tey
Copy link
Mannequin

Tey mannequin commented Jun 9, 2018

BPO 33817
Nosy @serhiy-storchaka, @miss-islington, @Tey
PRs
  • [2.7] bpo-33817: Fix _PyString_Resize() and _PyUnicode_Resize() for empty strings. #11515
  • [2.7] bpo-33817: Fix _PyString_Resize() and _PyUnicode_Resize() for empty strings. #11515
  • [2.7] bpo-33817: Fix _PyString_Resize() and _PyUnicode_Resize() for empty strings. #11515
  • bpo-33817: Fix _PyBytes_Resize() for empty bytes object. #11516
  • bpo-33817: Fix _PyBytes_Resize() for empty bytes object. #11516
  • bpo-33817: Fix _PyBytes_Resize() for empty bytes object. #11516
  • [3.7] bpo-33817: Fix _PyBytes_Resize() for empty bytes object. (GH-11516) #11532
  • [3.7] bpo-33817: Fix _PyBytes_Resize() for empty bytes object. (GH-11516) #11532
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
    closed_at = <Date 2019-01-12.07:48:00.217>
    created_at = <Date 2018-06-09.20:36:16.533>
    labels = ['interpreter-core', 'type-bug']
    title = 'PyString_FromFormatV() fails to build empty strings'
    updated_at = <Date 2019-01-12.07:48:00.217>
    user = 'https://github.com/Tey'

    bugs.python.org fields:

    activity = <Date 2019-01-12.07:48:00.217>
    actor = 'serhiy.storchaka'
    assignee = 'serhiy.storchaka'
    closed = True
    closed_date = <Date 2019-01-12.07:48:00.217>
    closer = 'serhiy.storchaka'
    components = ['Interpreter Core']
    creation = <Date 2018-06-09.20:36:16.533>
    creator = 'Tey'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 33817
    keywords = ['patch', 'patch', 'patch']
    message_count = 6.0
    messages = ['319180', '319226', '319227', '333518', '333519', '333520']
    nosy_count = 3.0
    nosy_names = ['serhiy.storchaka', 'miss-islington', 'Tey']
    pr_nums = ['11515', '11515', '11515', '11516', '11516', '11516', '11532', '11532']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue33817'
    versions = ['Python 2.7']

    @Tey
    Copy link
    Mannequin Author

    Tey mannequin commented Jun 9, 2018

    Making PyString_FromFormatV() build an empty string on 2.7 will fail and raise the following error:
    SystemError: Objects/stringobject.c:3903: bad argument to internal function

    It does not matter if format is empty or not (e.g., both PyString_FromFormat("") and PyString_FromFormat("%s", "")).

    The exception is raised from _PyString_Resize() (called at the end of PyString_FromFormatV()), because the string given to that method has a ref count different than 1. This is expected for empty strings, because PyString_FromStringAndSize() returns a reference to <nullstring> when <size> is 0.

    A possible fix would be to prevent the call to _PyString_Resize() from PyString_FromFormatV() if <string> is equal to <nullstring>, or if there is no need to resize the string.

    Python 3 versions before 3.6 might be impacted as well through PyBytes_FromFormatV() (cannot check).

    The following code can be used to trigger the bug:

        static int dobug(const char *fmt, ...) {
            va_list args;
            va_start(args, fmt);
        
            PyObject *str = PyString_FromFormatV(fmt, args);
            va_end(args);
        
            if(str == NULL) {
                fprintf(stderr, "Error: PyString_FromFormatV(%s) returned NULL\n", fmt);
                return -1;
            }
        
            Py_DECREF(str);
        
            return 0;
        }
    
        static PyObject* bug(PyObject *self) {
            fprintf(stderr, "dobug(\"\") => %d\n", dobug(""));
            fprintf(stderr, "dobug(\"%%s\", \"\") => %d\n", dobug("%s", ""));
        
            if(PyErr_Occurred()) return NULL;
            Py_RETURN_NONE;
        }

    @Tey Tey mannequin added interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Jun 9, 2018
    @serhiy-storchaka serhiy-storchaka self-assigned this Jun 10, 2018
    @Tey
    Copy link
    Mannequin Author

    Tey mannequin commented Jun 10, 2018

    For the record, it does not fail on 3.x because _PyBytes_Resize() checks if the "string" needs to be resized (and returns if not) before checking its ref count. Maybe something similar should be done in _PyString_Resize() for 2.x.

    @Tey
    Copy link
    Mannequin Author

    Tey mannequin commented Jun 10, 2018

    BTW, problem does appear in 3.4 as it's only been fixed in 3.5+ as a fix for issue bpo-25270

    @serhiy-storchaka
    Copy link
    Member

    New changeset 44cc482 by Serhiy Storchaka in branch 'master':
    bpo-33817: Fix _PyBytes_Resize() for empty bytes object. (GH-11516)
    44cc482

    @serhiy-storchaka
    Copy link
    Member

    New changeset 08a81df by Serhiy Storchaka in branch '2.7':
    bpo-33817: Fix _PyString_Resize() and _PyUnicode_Resize() for empty strings. (GH-11515)
    08a81df

    @miss-islington
    Copy link
    Contributor

    New changeset d39c192 by Miss Islington (bot) in branch '3.7':
    bpo-33817: Fix _PyBytes_Resize() for empty bytes object. (GH-11516)
    d39c192

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants