This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients pitrou, serhiy.storchaka, vstinner
Date 2013-11-13.23:22:06
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1384384926.73.0.738563316717.issue19568@psf.upfronthosting.co.za>
In-reply-to
Content
This issue is tricky, I will try to explain it.

To understand the bug, I wrote the following function:

static int
_PyByteArray_CheckConsistency(PyByteArrayObject *obj)
{
    assert(obj != NULL);
    assert(PyByteArray_Check(obj));

    assert(Py_SIZE(obj) >= 0);
    assert(obj->ob_bytes <= obj->ob_start);
    assert((obj->ob_start - obj->ob_bytes) <= obj->ob_alloc);
    assert((obj->ob_alloc - (obj->ob_start - obj->ob_bytes)) >= Py_SIZE(obj));
    return 1;
}


In this issue, I'm concerned by bytearray_setslice_linear() with growth < 0. There are two cases:

(A) lo == 0
(B) lo != 0.


(A) It's trivial to rollback the change: restore previous value of ob_start. (Another option is to update the size, which should be the same if I understood correctly.) You retrieve the original object.

Expected result:

>>> b=bytearray(b'1234567890')
>>> del b[:6]
>>> b
bytearray(b'7890')

Current behaviour on MemoryError

>>> b=bytearray(b'1234567890')
>>> del b[:6]
>>> b
bytearray(b'7890\x00\xfb\xfb\xfb\xfb\xfb')

With the patch:

>>> b=bytearray(b'1234567890')
>>> del b[:6]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError
>>> b
bytearray(b'1234567890')



(B) You cannot restore removed bytes. Currently, you get an inconsistent bytearray object because its content is updated but not its size.

No error:

>>> b=bytearray(b'1234567890')
>>> del b[3:6]
>>> b
bytearray(b'1237890')

Current behaviour on MemoryError:

>>> b=bytearray(b'1234567890')
>>> del b[3:6]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError
>>> b
bytearray(b'1237890890')

With the patch:

>>> b=bytearray(b'1234567890')
>>> del b[3:6]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError
>>> b
bytearray(b'1237890')

With the patch, the deletion succeeded even if you get a MemoryError. The bytearray object is consistent. It's just that its buffer could be smaller (it wastes memory).


Note: I used gdb to inject a MemoryError in PyByteArray_Resize().
History
Date User Action Args
2013-11-13 23:22:06vstinnersetrecipients: + vstinner, pitrou, serhiy.storchaka
2013-11-13 23:22:06vstinnersetmessageid: <1384384926.73.0.738563316717.issue19568@psf.upfronthosting.co.za>
2013-11-13 23:22:06vstinnerlinkissue19568 messages
2013-11-13 23:22:06vstinnercreate