classification
Title: Incorrect assert in str_subtype_new
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.6, Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: Kevin Modzelewski, mark.dickinson, python-dev, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2015-07-26 23:21 by Kevin Modzelewski, last changed 2015-12-01 23:25 by Kevin Modzelewski. This issue is now closed.

Files
File name Uploaded Description Edit
issue24731.patch serhiy.storchaka, 2015-11-22 23:11 Patch for 3.6 review
issue24731-2.7.patch serhiy.storchaka, 2015-11-22 23:11 Patch for 2.7 review
Messages (5)
msg247451 - (view) Author: Kevin Modzelewski (Kevin Modzelewski) Date: 2015-07-26 23:21
(Using python 3 terminology)  str_subtype_new is the function that creates instances of any subtypes of bytes (ie is called by bytes_new if the requested type is not PyBytes_Type -- looks like this function's name comes from python 2).  Its approach is to create a bytes object using the same arguments, and then copy the resulting data into the subclass-instance's memory.  It does

    tmp = bytes_new(&PyBytes_Type, args, kwds);
    [error checking]
    assert(PyBytes_CheckExact(tmp));

The problem is that bytes_new can return a subclass of bytes, if the argument provides a __bytes__ method that returns a bytes-subtype.  For example

    class MyBytes(bytes):
        pass
    class C(object):
        def __bytes__(self):
            return MyBytes(b"hello world")
    MyBytes(C()) # fails the assert

This doesn't seem to cause any issues other than the failing assert in debug builds; it seems like the assert should just be relaxed from PyBytes_CheckExact to PyBytes_Check since that's enough to guarantee that the upcoming manipulation of the "tmp" variable is going to be valid.  Also, this would match how unicode_subtype_new behaves.

This bug also applies to Python 2, since I think the relevant code is the same, though in that case it applies to str instead of bytes.
msg255117 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-11-22 23:11
Thank you for your report Kevin and sorry for the delay. I confirm the bug and agree with your solution. The same bug exists for ints and floats.

Here are patches that fix crashes.
msg255351 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-11-25 13:56
New changeset d8e0b54ece62 by Serhiy Storchaka in branch '3.4':
Issue #24731: Fixed crash on converting objects with special methods
https://hg.python.org/cpython/rev/d8e0b54ece62

New changeset 80efe5cc8934 by Serhiy Storchaka in branch '3.5':
Issue #24731: Fixed crash on converting objects with special methods
https://hg.python.org/cpython/rev/80efe5cc8934

New changeset 1c4d256cc370 by Serhiy Storchaka in branch 'default':
Issue #24731: Fixed crash on converting objects with special methods
https://hg.python.org/cpython/rev/1c4d256cc370

New changeset 37158c067b25 by Serhiy Storchaka in branch '2.7':
Issue #24731: Fixed crash on converting objects with special methods
https://hg.python.org/cpython/rev/37158c067b25
msg255583 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-11-29 18:14
New changeset 2ea1a3bf448f by Serhiy Storchaka in branch '2.7':
Fixed Py3k warnings in tests for issue #24731.
https://hg.python.org/cpython/rev/2ea1a3bf448f
msg255683 - (view) Author: Kevin Modzelewski (Kevin Modzelewski) Date: 2015-12-01 23:25
Awesome, thanks!
History
Date User Action Args
2015-12-01 23:25:40Kevin Modzelewskisetmessages: + msg255683
2015-11-29 18:14:17python-devsetmessages: + msg255583
2015-11-25 13:57:54serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2015-11-25 13:56:49python-devsetnosy: + python-dev
messages: + msg255351
2015-11-22 23:11:59serhiy.storchakasetfiles: + issue24731-2.7.patch
2015-11-22 23:11:11serhiy.storchakasetfiles: + issue24731.patch

nosy: + mark.dickinson
messages: + msg255117

keywords: + patch
stage: needs patch -> patch review
2015-07-27 04:33:54serhiy.storchakasetversions: + Python 3.4, Python 3.6
nosy: + serhiy.storchaka

assignee: serhiy.storchaka
type: crash
stage: needs patch
2015-07-26 23:21:25Kevin Modzelewskicreate