Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python 3: Segfault instead of MemoryError when bytearray too big #66531

Closed
swanson mannequin opened this issue Sep 4, 2014 · 10 comments
Closed

Python 3: Segfault instead of MemoryError when bytearray too big #66531

swanson mannequin opened this issue Sep 4, 2014 · 10 comments
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@swanson
Copy link
Mannequin

swanson mannequin commented Sep 4, 2014

BPO 22335
Nosy @pitrou, @vstinner, @jwilk, @ezio-melotti, @serhiy-storchaka
Files
  • issue19087.patch
  • ba_resize.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2014-11-02.17:43:23.082>
created_at = <Date 2014-09-04.10:38:57.101>
labels = ['interpreter-core', 'type-crash']
title = 'Python 3: Segfault instead of MemoryError when bytearray too big'
updated_at = <Date 2014-11-02.17:43:23.080>
user = 'https://bugs.python.org/swanson'

    bugs.python.org fields:

    activity = <Date 2014-11-02.17:43:23.080>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2014-11-02.17:43:23.082>
closer = 'pitrou'
components = ['Interpreter Core']
creation = <Date 2014-09-04.10:38:57.101>
creator = 'swanson'
dependencies = []
files = ['36917', '37108']
hgrepos = []
issue_num = 22335
keywords = ['patch']
message_count = 10.0
messages = ['226356', '226358', '229314', '229322', '230496', '230497', '230499', '230505', '230509', '230510']
nosy_count = 8.0
nosy_names = ['pitrou', 'vstinner', 'jwilk', 'ezio.melotti', 'Arfrever', 'python-dev', 'serhiy.storchaka', 'swanson']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue22335'
versions = ['Python 3.4', 'Python 3.5']
    @swanson
    Copy link
    Mannequin Author

    swanson mannequin commented Sep 4, 2014

    On Python 3, but not Python 2, you crash with a Segmentation Fault instead of getting a MemoryError as expected. It seems to only be a problem with bytearray, not with other things like tuple:

    $ python3
Python 3.4.0 (default, Apr 11 2014, 13:05:18) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> bytearray(0x7FFFFFFF)
Segmentation fault (core dumped)
$

    compare to:

    $ python
Python 2.7.6 (default, Mar 22 2014, 22:59:38) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> bytearray(0x7FFFFFFF)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError
>>> 


$ python3
Python 3.4.0 (default, Apr 11 2014, 13:05:18) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> (0,)*0x7FFFFFFF
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError
>>>

    @swanson swanson mannequin added the type-crash A hard crash of the interpreter, possibly with a core dump label Sep 4, 2014
    @serhiy-storchaka
    Copy link
    Member

    Looks as integer overflow introduced in bpo-19087.

    + if (size + logical_offset + 1 < alloc) {

    @jwilk
    Copy link
    Mannequin

    jwilk mannequin commented Oct 14, 2014

    This patch should fix it.

    @jwilk
    Copy link
    Mannequin

    jwilk mannequin commented Oct 14, 2014

    On a second thought, "logical_offset + 1" alone could overflow; and there are apparently other possible integer overflows in this function.

    @serhiy-storchaka
    Copy link
    Member

    "logical_offset + 1" can't overflow because logical_offset is an offset in allocated array not counting final null byte.

    @pitrou
    Copy link
    Member

    pitrou commented Nov 2, 2014

    Note that there are two possible crashes in debug mode:

    $ ./python -c "bytearray(2**31-1)"
Erreur de segmentation

$ ./python -c "bytearray(2**31-2)"
python: Objects/obmalloc.c:1179: _PyObject_Alloc: Assertion `nelem <= ((Py_ssize_t)(((size_t)-1)>>1)) / elsize' failed.
Abandon

    @pitrou
    Copy link
    Member

    pitrou commented Nov 2, 2014

    Here is a patch. It also fixes a bug in the debug allocators, which didn't properly check for Py_ssize_t overflow.

    @pitrou pitrou added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Nov 2, 2014
    @serhiy-storchaka
    Copy link
    Member

    LGTM.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Nov 2, 2014

    New changeset 1590c594550e by Antoine Pitrou in branch '3.4':
    Issue bpo-22335: Fix crash when trying to enlarge a bytearray to 0x7fffffff bytes on a 32-bit platform.
    https://hg.python.org/cpython/rev/1590c594550e

    New changeset f0b334ae95c9 by Antoine Pitrou in branch 'default':
    Issue bpo-22335: Fix crash when trying to enlarge a bytearray to 0x7fffffff bytes on a 32-bit platform.
    https://hg.python.org/cpython/rev/f0b334ae95c9

    @pitrou
    Copy link
    Member

    pitrou commented Nov 2, 2014

    Thank you. This is now pushed.

    @pitrou pitrou closed this as completed Nov 2, 2014
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @pitrou @serhiy-storchaka