➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Invalid read in test_codecs
Type: behavior Stage: resolved
Components: Interpreter Core, Unicode Versions: Python 3.2, Python 3.3, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: ezio.melotti, python-dev, serhiy.storchaka, skrah
Priority: normal Keywords: patch

Created on 2013-01-26 20:41 by skrah, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
decodeunicodeinternal_overflow-2.7.patch serhiy.storchaka, 2013-01-26 22:06 review
decodeunicodeinternal_overflow-3.2.patch serhiy.storchaka, 2013-01-26 22:06 review
decodeunicodeinternal_overflow-3.3.patch serhiy.storchaka, 2013-01-26 22:06 review
decodeunicodeinternal_overflow-3.4.patch serhiy.storchaka, 2013-01-26 22:06 review
Messages (5)
msg180709 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-01-26 20:41 
Found this in test_codecs running under Valgrind (Python 3.3):


test_bug1251300 (test.test_codecs.UnicodeInternalTest) ... ==11511== Invalid read of size 1
==11511==    at 0x44AF37: _PyUnicode_DecodeUnicodeInternal (unicodeobject.c:6133)
==11511==    by 0x4DEB5C: unicode_internal_decode (_codecsmodule.c:251)
==11511==    by 0x5093F6: PyObject_Call (abstract.c:2082)
==11511==    by 0x47D7F2: PyEval_CallObjectWithKeywords (ceval.c:3942)
==11511==    by 0x491C38: PyCodec_Decode (codecs.c:403)
==11511==    by 0x459D7D: PyUnicode_Decode (unicodeobject.c:3129)
==11511==    by 0x45A287: PyUnicode_FromEncodedObject (unicodeobject.c:3023)
==11511==    by 0x519A45: bytes_decode (bytesobject.c:2320)
==11511==    by 0x484AB8: PyEval_EvalFrameEx (ceval.c:4374)
==11511==    by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150)
==11511==    by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
==11511==    by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160)
==11511==  Address 0x984a7e2 is 0 bytes after a block of size 34 alloc'd
==11511==    at 0x4C27972: realloc (vg_replace_malloc.c:525)
==11511==    by 0x51AC34: _PyBytes_Resize (bytesobject.c:2881)
==11511==    by 0x51B1FA: PyBytes_FromObject (bytesobject.c:2732)
==11511==    by 0x51C134: bytes_new (bytesobject.c:2594)
==11511==    by 0x42A4E4: type_call (typeobject.c:723)
==11511==    by 0x5093F6: PyObject_Call (abstract.c:2082)
==11511==    by 0x4843D5: PyEval_EvalFrameEx (ceval.c:4282)
==11511==    by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150)
==11511==    by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
==11511==    by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160)
==11511==    by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
==11511==    by 0x538EF8: function_call (funcobject.c:633)
==11511== 



_PyUnicode_DecodeUnicodeInternal (s=0x984a7e0 "", size=<value optimized out>, errors=0x0)
    at Objects/unicodeobject.c:6133
6133            ((char *) &uch)[2] = s[2];




==================================================================




==11511== 
==11511== Debugger has detached.  Valgrind regains control.  We continue.
==11511== Invalid read of size 1
==11511==    at 0x44AF3E: _PyUnicode_DecodeUnicodeInternal (unicodeobject.c:6134)
==11511==    by 0x4DEB5C: unicode_internal_decode (_codecsmodule.c:251)
==11511==    by 0x5093F6: PyObject_Call (abstract.c:2082)
==11511==    by 0x47D7F2: PyEval_CallObjectWithKeywords (ceval.c:3942)
==11511==    by 0x491C38: PyCodec_Decode (codecs.c:403)
==11511==    by 0x459D7D: PyUnicode_Decode (unicodeobject.c:3129)
==11511==    by 0x45A287: PyUnicode_FromEncodedObject (unicodeobject.c:3023)
==11511==    by 0x519A45: bytes_decode (bytesobject.c:2320)
==11511==    by 0x484AB8: PyEval_EvalFrameEx (ceval.c:4374)
==11511==    by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150)
==11511==    by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
==11511==    by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160)
==11511==  Address 0x984a7e3 is 1 bytes after a block of size 34 alloc'd
==11511==    at 0x4C27972: realloc (vg_replace_malloc.c:525)
==11511==    by 0x51AC34: _PyBytes_Resize (bytesobject.c:2881)
==11511==    by 0x51B1FA: PyBytes_FromObject (bytesobject.c:2732)
==11511==    by 0x51C134: bytes_new (bytesobject.c:2594)
==11511==    by 0x42A4E4: type_call (typeobject.c:723)
==11511==    by 0x5093F6: PyObject_Call (abstract.c:2082)
==11511==    by 0x4843D5: PyEval_EvalFrameEx (ceval.c:4282)
==11511==    by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150)
==11511==    by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
==11511==    by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160)
==11511==    by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
==11511==    by 0x538EF8: function_call (funcobject.c:633)
==11511== 


Loaded symbols for /usr/lib/gconv/ISO8859-9.so
_PyUnicode_DecodeUnicodeInternal (s=0x8295790 "", size=<value optimized out>, errors=0x0)
    at Objects/unicodeobject.c:6134
6134            ((char *) &uch)[3] = s[3];
msg180711 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-01-26 21:02 
Same in test_codeccallbacks:

test_badhandlerresults (test.test_codeccallbacks.CodecCallbackTest) ... ==11604== Invalid read of size 1
==11604==    at 0x44AF37: _PyUnicode_DecodeUnicodeInternal (unicodeobject.c:6133)
==11604==    by 0x4DEB5C: unicode_internal_decode (_codecsmodule.c:251)
==11604==    by 0x5093F6: PyObject_Call (abstract.c:2082)
==11604==    by 0x47D7F2: PyEval_CallObjectWithKeywords (ceval.c:3942)
==11604==    by 0x491C38: PyCodec_Decode (codecs.c:403)
==11604==    by 0x459D7D: PyUnicode_Decode (unicodeobject.c:3129)
==11604==    by 0x45A287: PyUnicode_FromEncodedObject (unicodeobject.c:3023)
==11604==    by 0x519A45: bytes_decode (bytesobject.c:2320)
==11604==    by 0x484AB8: PyEval_EvalFrameEx (ceval.c:4374)
==11604==    by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150)
==11604==    by 0x486779: PyEval_EvalCodeEx (ceval.c:3433)
==11604==    by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160)
==11604==  Address 0xfa1f8a2 is 0 bytes after a block of size 34 alloc'd
==11604==    at 0x4C27972: realloc (vg_replace_malloc.c:525)
==11604==    by 0x51AC34: _PyBytes_Resize (bytesobject.c:2881)
==11604==    by 0x51C338: PyBytes_DecodeEscape (bytesobject.c:495)
==11604==    by 0x56E871: ast_for_expr (ast.c:3837)
==11604==    by 0x570562: ast_for_testlist (ast.c:1106)
==11604==    by 0x56E859: ast_for_expr (ast.c:1881)
==11604==    by 0x570562: ast_for_testlist (ast.c:1106)
==11604==    by 0x56E859: ast_for_expr (ast.c:1881)
==11604==    by 0x5715C4: ast_for_stmt (ast.c:3302)
==11604==    by 0x5724F8: ast_for_suite (ast.c:3086)
==11604==    by 0x5715E3: ast_for_stmt (ast.c:3305)
==11604==    by 0x5724F8: ast_for_suite (ast.c:3086)

_PyUnicode_DecodeUnicodeInternal (s=0xfa1f8a0 "", size=<value optimized out>, errors=
    0xf652fa0 "test.badhandler") at Objects/unicodeobject.c:6133
6133            ((char *) &uch)[2] = s[2];



[...]
_PyUnicode_DecodeUnicodeInternal (s=0xfa1f8a0 "", size=<value optimized out>, errors=
    0xf652fa0 "test.badhandler") at Objects/unicodeobject.c:6134
6134            ((char *) &uch)[3] = s[3];
msg180716 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-26 22:06 
Here are patches for all 4 versions.
msg181437 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-02-05 14:07 
Ping.
msg181610 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-02-07 14:30 
New changeset 498b54e0e856 by Serhiy Storchaka in branch '2.7':
Issue #17043: The unicode-internal decoder no longer read past the end of
http://hg.python.org/cpython/rev/498b54e0e856

New changeset 0f1c2e2b6bc2 by Serhiy Storchaka in branch '3.2':
Issue #17043: The unicode-internal decoder no longer read past the end of
http://hg.python.org/cpython/rev/0f1c2e2b6bc2

New changeset fec2976c8503 by Serhiy Storchaka in branch '3.3':
Issue #17043: The unicode-internal decoder no longer read past the end of
http://hg.python.org/cpython/rev/fec2976c8503

New changeset eb0370d4686c by Serhiy Storchaka in branch 'default':
Issue #17043: The unicode-internal decoder no longer read past the end of
http://hg.python.org/cpython/rev/eb0370d4686c
History
Date User Action Args
2022-04-11 14:57:41adminsetgithub: 61245
2013-02-09 20:45:31serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2013-02-07 14:30:47python-devsetnosy: + python-dev
messages: + msg181610
2013-02-05 14:07:29serhiy.storchakasetmessages: + msg181437
2013-01-31 14:38:51serhiy.storchakasetassignee: serhiy.storchaka
2013-01-26 22:07:47serhiy.storchakasetversions: + Python 2.7, Python 3.2, Python 3.4
nosy: + ezio.melotti

components: + Interpreter Core, Unicode
type: behavior
stage: patch review
2013-01-26 22:06:44serhiy.storchakasetfiles: + decodeunicodeinternal_overflow-2.7.patch, decodeunicodeinternal_overflow-3.2.patch, decodeunicodeinternal_overflow-3.3.patch, decodeunicodeinternal_overflow-3.4.patch
keywords: + patch
messages: + msg180716
2013-01-26 21:02:36skrahsetmessages: + msg180711
2013-01-26 20:41:15skrahcreate