This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients gregory.p.smith, neologix, python-dev, serhiy.storchaka, vstinner
Date 2013-07-17.10:12:14
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <CAMpsgwY+NeRjUbNA+9pHK_LbWJsrYMnfsjVdZW0kwMBMDH70mg@mail.gmail.com>
In-reply-to <1374046384.85.0.362540228458.issue18408@psf.upfronthosting.co.za>
Content
2013/7/17 Serhiy Storchaka <report@bugs.python.org>:
> listpop.patch:
>
> list_resize(a, Py_SIZE(a) + d) for d < 0 fails only when PyMem_Realloc(p, n) returns NULL if n < allocated size. Is it possible? Is it desired behavior? Perhaps we should declare that PyMem_Realloc() should return the original pointer if it can't shrink an allocated block.

Resizing a buffer to make it smaller "should" not fail, and if it
fails, the case is very unlikely. But I prefer to handle the case,
rather than hoping that the case cannot occur in the libc
implementation.

For example, _PyObject_Realloc() of pymalloc allocates a *new* buffer
to shrink an existing buffer if the size is reduced by more than 25%:

        if (nbytes <= size) {
            /* The block is staying the same or shrinking.  If
             * it's shrinking, there's a tradeoff:  it costs
             * cycles to copy the block to a smaller size class,
             * but it wastes memory not to copy it.  The
             * compromise here is to copy on shrink only if at
             * least 25% of size can be shaved off.
             */
            if (4 * nbytes > 3 * size) {
                /* It's the same,
                 * or shrinking and new/old > 3/4.
                 */
                return p;
            }
            size = nbytes;
        }
        bp = _PyObject_Malloc(ctx, nbytes);

I plan also to test Python with PyMem_Malloc using pymalloc to check
if it's faster or not. So PyMem_Realloc() may fail when shrinking a
buffer ;-)

By the way, I already fixed similar bugs (handle PyMem_Realloc failure
when shrinking the buffer) in this issue. Examples:
http://hg.python.org/cpython/rev/68887e177dd4
http://hg.python.org/cpython/rev/549d8d3297f2

listpop.patch only changes the error handling to make it more
resilient when the "very unlikely" case occurs ;-)
History
Date User Action Args
2013-07-17 10:12:14vstinnersetrecipients: + vstinner, gregory.p.smith, neologix, python-dev, serhiy.storchaka
2013-07-17 10:12:14vstinnerlinkissue18408 messages
2013-07-17 10:12:14vstinnercreate