This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Calling _PyBytes_Resize() on 1-byte bytes may raise error
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: malin
Priority: normal Keywords:

Created on 2021-01-26 03:52 by malin, last changed 2022-04-11 14:59 by admin.

Messages (1)
msg385689 - (view) Author: Ma Lin (malin) * Date: 2021-01-26 03:52
PyBytes_FromStringAndSize() uses a global cache for 1-byte bytes:
https://github.com/python/cpython/blob/v3.10.0a4/Objects/bytesobject.c#L147

    if (size == 1 && str != NULL) {
        struct _Py_bytes_state *state = get_bytes_state();
        op = state->characters[*str & UCHAR_MAX];
        if (op != NULL) {
            Py_INCREF(op);
            return (PyObject *)op;
        }
    }

_PyBytes_Resize() will raise an error when (refcount != 1):
https://github.com/python/cpython/blob/v3.10.0a4/Objects/bytesobject.c#L3029

Then this code will raise an exception:

    obj1 = PyBytes_FromStringAndSize("a", 1);
    obj2 = PyBytes_FromStringAndSize("a", 1);
    ret = _PyBytes_Resize(&obj2, 2);  // ret is -1

BTW, 0-byte bytes comes from a global singleton, but _PyBytes_Resize() processes it before checking refcount:
https://github.com/python/cpython/blob/v3.10.0a4/Objects/bytesobject.c#L3021

    if (Py_SIZE(v) == 0) {
        if (newsize == 0) {
            return 0;
        }
        *pv = _PyBytes_FromSize(newsize, 0);
        Py_DECREF(v);
        return (*pv == NULL) ? -1 : 0;
    }
    if (Py_REFCNT(v) != 1) {
        goto error;
    }

_PyBytes_Resize() doc:
    
    Only use this to build up a brand new bytes object; don’t use this if the
    bytes may already be known in other parts of the code. It is an error to
    call this function if the refcount on the input bytes object is not one.
History
Date User Action Args
2022-04-11 14:59:40adminsetgithub: 87193
2021-01-26 03:52:05malincreate