Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Invalid read in test_codecs #61245

Closed
skrah mannequin opened this issue Jan 26, 2013 · 5 comments
Closed

Invalid read in test_codecs #61245

skrah mannequin opened this issue Jan 26, 2013 · 5 comments
Assignees
@serhiy-storchaka
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-unicode type-bug An unexpected behavior, bug, or error

Comments

@skrah
Copy link
Mannequin

skrah mannequin commented Jan 26, 2013

BPO 17043
Nosy @ezio-melotti, @skrah, @serhiy-storchaka
Files
  • decodeunicodeinternal_overflow-2.7.patch
  • decodeunicodeinternal_overflow-3.2.patch
  • decodeunicodeinternal_overflow-3.3.patch
  • decodeunicodeinternal_overflow-3.4.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2013-02-09.20:45:31.705>
created_at = <Date 2013-01-26.20:41:15.433>
labels = ['interpreter-core', 'type-bug', 'expert-unicode']
title = 'Invalid read in test_codecs'
updated_at = <Date 2013-02-09.20:45:31.704>
user = 'https://github.com/skrah'

    bugs.python.org fields:

    activity = <Date 2013-02-09.20:45:31.704>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2013-02-09.20:45:31.705>
closer = 'serhiy.storchaka'
components = ['Interpreter Core', 'Unicode']
creation = <Date 2013-01-26.20:41:15.433>
creator = 'skrah'
dependencies = []
files = ['28860', '28861', '28862', '28863']
hgrepos = []
issue_num = 17043
keywords = ['patch']
message_count = 5.0
messages = ['180709', '180711', '180716', '181437', '181610']
nosy_count = 4.0
nosy_names = ['ezio.melotti', 'skrah', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue17043'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4']
    @skrah
    Copy link
    Mannequin Author

    skrah mannequin commented Jan 26, 2013

    Found this in test_codecs running under Valgrind (Python 3.3):

    test_bug1251300 (test.test_codecs.UnicodeInternalTest) ... ==11511== Invalid read of size 1
    ==11511== at 0x44AF37: _PyUnicode_DecodeUnicodeInternal (unicodeobject.c:6133)
    ==11511== by 0x4DEB5C: unicode_internal_decode (_codecsmodule.c:251)
    ==11511== by 0x5093F6: PyObject_Call (abstract.c:2082)
    ==11511== by 0x47D7F2: PyEval_CallObjectWithKeywords (ceval.c:3942)
    ==11511== by 0x491C38: PyCodec_Decode (codecs.c:403)
    ==11511== by 0x459D7D: PyUnicode_Decode (unicodeobject.c:3129)
    ==11511== by 0x45A287: PyUnicode_FromEncodedObject (unicodeobject.c:3023)
    ==11511== by 0x519A45: bytes_decode (bytesobject.c:2320)
    ==11511== by 0x484AB8: PyEval_EvalFrameEx (ceval.c:4374)
    ==11511== by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150)
    ==11511== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
    ==11511== by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160)
    ==11511== Address 0x984a7e2 is 0 bytes after a block of size 34 alloc'd
    ==11511== at 0x4C27972: realloc (vg_replace_malloc.c:525)
    ==11511== by 0x51AC34: _PyBytes_Resize (bytesobject.c:2881)
    ==11511== by 0x51B1FA: PyBytes_FromObject (bytesobject.c:2732)
    ==11511== by 0x51C134: bytes_new (bytesobject.c:2594)
    ==11511== by 0x42A4E4: type_call (typeobject.c:723)
    ==11511== by 0x5093F6: PyObject_Call (abstract.c:2082)
    ==11511== by 0x4843D5: PyEval_EvalFrameEx (ceval.c:4282)
    ==11511== by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150)
    ==11511== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
    ==11511== by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160)
    ==11511== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
    ==11511== by 0x538EF8: function_call (funcobject.c:633)
    ==11511==

    _PyUnicode_DecodeUnicodeInternal (s=0x984a7e0 "", size=<value optimized out>, errors=0x0)
    at Objects/unicodeobject.c:6133
    6133 ((char *) &uch)[2] = s[2];

    ==================================================================

    ==11511==
    ==11511== Debugger has detached. Valgrind regains control. We continue.
    ==11511== Invalid read of size 1
    ==11511== at 0x44AF3E: _PyUnicode_DecodeUnicodeInternal (unicodeobject.c:6134)
    ==11511== by 0x4DEB5C: unicode_internal_decode (_codecsmodule.c:251)
    ==11511== by 0x5093F6: PyObject_Call (abstract.c:2082)
    ==11511== by 0x47D7F2: PyEval_CallObjectWithKeywords (ceval.c:3942)
    ==11511== by 0x491C38: PyCodec_Decode (codecs.c:403)
    ==11511== by 0x459D7D: PyUnicode_Decode (unicodeobject.c:3129)
    ==11511== by 0x45A287: PyUnicode_FromEncodedObject (unicodeobject.c:3023)
    ==11511== by 0x519A45: bytes_decode (bytesobject.c:2320)
    ==11511== by 0x484AB8: PyEval_EvalFrameEx (ceval.c:4374)
    ==11511== by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150)
    ==11511== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
    ==11511== by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160)
    ==11511== Address 0x984a7e3 is 1 bytes after a block of size 34 alloc'd
    ==11511== at 0x4C27972: realloc (vg_replace_malloc.c:525)
    ==11511== by 0x51AC34: _PyBytes_Resize (bytesobject.c:2881)
    ==11511== by 0x51B1FA: PyBytes_FromObject (bytesobject.c:2732)
    ==11511== by 0x51C134: bytes_new (bytesobject.c:2594)
    ==11511== by 0x42A4E4: type_call (typeobject.c:723)
    ==11511== by 0x5093F6: PyObject_Call (abstract.c:2082)
    ==11511== by 0x4843D5: PyEval_EvalFrameEx (ceval.c:4282)
    ==11511== by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150)
    ==11511== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
    ==11511== by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160)
    ==11511== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
    ==11511== by 0x538EF8: function_call (funcobject.c:633)
    ==11511==

    Loaded symbols for /usr/lib/gconv/ISO8859-9.so
    _PyUnicode_DecodeUnicodeInternal (s=0x8295790 "", size=<value optimized out>, errors=0x0)
    at Objects/unicodeobject.c:6134
    6134 ((char *) &uch)[3] = s[3];

    @skrah
    Copy link
    Mannequin Author

    skrah mannequin commented Jan 26, 2013

    Same in test_codeccallbacks:

    test_badhandlerresults (test.test_codeccallbacks.CodecCallbackTest) ... ==11604== Invalid read of size 1
    ==11604== at 0x44AF37: _PyUnicode_DecodeUnicodeInternal (unicodeobject.c:6133)
    ==11604== by 0x4DEB5C: unicode_internal_decode (_codecsmodule.c:251)
    ==11604== by 0x5093F6: PyObject_Call (abstract.c:2082)
    ==11604== by 0x47D7F2: PyEval_CallObjectWithKeywords (ceval.c:3942)
    ==11604== by 0x491C38: PyCodec_Decode (codecs.c:403)
    ==11604== by 0x459D7D: PyUnicode_Decode (unicodeobject.c:3129)
    ==11604== by 0x45A287: PyUnicode_FromEncodedObject (unicodeobject.c:3023)
    ==11604== by 0x519A45: bytes_decode (bytesobject.c:2320)
    ==11604== by 0x484AB8: PyEval_EvalFrameEx (ceval.c:4374)
    ==11604== by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150)
    ==11604== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
    ==11604== by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160)
    ==11604== Address 0xfa1f8a2 is 0 bytes after a block of size 34 alloc'd
    ==11604== at 0x4C27972: realloc (vg_replace_malloc.c:525)
    ==11604== by 0x51AC34: _PyBytes_Resize (bytesobject.c:2881)
    ==11604== by 0x51C338: PyBytes_DecodeEscape (bytesobject.c:495)
    ==11604== by 0x56E871: ast_for_expr (ast.c:3837)
    ==11604== by 0x570562: ast_for_testlist (ast.c:1106)
    ==11604== by 0x56E859: ast_for_expr (ast.c:1881)
    ==11604== by 0x570562: ast_for_testlist (ast.c:1106)
    ==11604== by 0x56E859: ast_for_expr (ast.c:1881)
    ==11604== by 0x5715C4: ast_for_stmt (ast.c:3302)
    ==11604== by 0x5724F8: ast_for_suite (ast.c:3086)
    ==11604== by 0x5715E3: ast_for_stmt (ast.c:3305)
    ==11604== by 0x5724F8: ast_for_suite (ast.c:3086)

    _PyUnicode_DecodeUnicodeInternal (s=0xfa1f8a0 "", size=<value optimized out>, errors=
    0xf652fa0 "test.badhandler") at Objects/unicodeobject.c:6133
    6133 ((char *) &uch)[2] = s[2];

    [...]
    _PyUnicode_DecodeUnicodeInternal (s=0xfa1f8a0 "", size=<value optimized out>, errors=
    0xf652fa0 "test.badhandler") at Objects/unicodeobject.c:6134
    6134 ((char *) &uch)[3] = s[3];

    @serhiy-storchaka
    Copy link
    Member

    Here are patches for all 4 versions.

    @serhiy-storchaka serhiy-storchaka added interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-unicode type-bug An unexpected behavior, bug, or error labels Jan 26, 2013
    @serhiy-storchaka serhiy-storchaka self-assigned this Jan 31, 2013
    @serhiy-storchaka
    Copy link
    Member

    Ping.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Feb 7, 2013

    New changeset 498b54e0e856 by Serhiy Storchaka in branch '2.7':
    Issue bpo-17043: The unicode-internal decoder no longer read past the end of
    http://hg.python.org/cpython/rev/498b54e0e856

    New changeset 0f1c2e2b6bc2 by Serhiy Storchaka in branch '3.2':
    Issue bpo-17043: The unicode-internal decoder no longer read past the end of
    http://hg.python.org/cpython/rev/0f1c2e2b6bc2

    New changeset fec2976c8503 by Serhiy Storchaka in branch '3.3':
    Issue bpo-17043: The unicode-internal decoder no longer read past the end of
    http://hg.python.org/cpython/rev/fec2976c8503

    New changeset eb0370d4686c by Serhiy Storchaka in branch 'default':
    Issue bpo-17043: The unicode-internal decoder no longer read past the end of
    http://hg.python.org/cpython/rev/eb0370d4686c

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @serhiy-storchaka serhiy-storchaka

    Labels
    interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-unicode type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @serhiy-storchaka