This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: SystemError in encoder
Type: behavior Stage:
Components: Interpreter Core, Unicode Versions: Python 3.1
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, benjamin.peterson, glynnc, jcsalterego, loewis
Priority: normal Keywords: patch

Created on 2009-06-29 15:31 by glynnc, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
issue6373.patch amaury.forgeotdarc, 2009-06-29 16:17
Messages (6)
msg89852 - (view) Author: Glynn Clements (glynnc) Date: 2009-06-29 15:31
Test case:

> "\udce4\udceb\udcef\udcf6\udcfc".encode("iso-8859-1", "surrogateescape")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: Objects/bytesobject.c:3182: bad argument to internal function

The line number corresponds to _PyBytes_Resize()
msg89853 - (view) Author: Jerry Chen (jcsalterego) Date: 2009-06-29 15:45
gdb trace

Breakpoint 1, _PyBytes_Resize (pv=0xbfffec3c, newsize=-495723) at
Objects/bytesobject.c:3179
3179		if (!PyBytes_Check(v) || Py_REFCNT(v) != 1 || newsize < 0) {
(gdb) where
#0  _PyBytes_Resize (pv=0xbfffec3c, newsize=-495723) at
Objects/bytesobject.c:3179
#1  0x0006fc07 in unicode_encode_ucs1 (p=0x3c3b8a, size=-1073746884,
errors=0x441ad0 "surrogateescape", limit=256) at
Objects/unicodeobject.c:4253
#2  0x000fafe7 in latin_1_encode (self=0x334490, args=0xbfffec3c) at
_codecsmodule.c:939
#3  0x000084a9 in PyObject_Call (func=0x3419b8, arg=0x3c9d28, kw=0x0) at
Objects/abstract.c:2160
#4  0x000a203e in PyEval_CallObjectWithKeywords (func=0x3419b8,
arg=0x441ae0, kw=0xbfffec3c) at Python/ceval.c:3694
#5  0x000b6c3b in PyCodec_Encode (object=0xbfffec3c, encoding=0xbfffec3c
"?\032D", errors=0x441d70 "surrogateescape") at Python/codecs.c:354
#6  0x000701c6 in PyUnicodeUCS2_AsEncodedString (unicode=0x4497c0,
encoding=0x441a30 "iso-8859-1", errors=0x441d70 "surrogateescape") at
Objects/unicodeobject.c:1420
#7  0x0007038c in unicode_encode (self=0x4497c0, args=0xbfffec3c) at
Objects/unicodeobject.c:7150
#8  0x000a71d8 in PyEval_EvalFrameEx (f=0x22e590, throwflag=0) at
Python/ceval.c:3814
#9  0x000a8beb in PyEval_EvalCodeEx (co=0x3aa698, globals=0x34c270,
locals=0x34c270, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0,
defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:3237
#10 0x000a8dbf in PyEval_EvalCode (co=0xbfffec3c, globals=0xbfffec3c,
locals=0xbfffec3c) at Python/ceval.c:651
#11 0x000cf473 in PyRun_InteractiveOneFlags (fp=0xa02e15e0,
filename=0x123994 "<stdin>", flags=0xbffff77c) at Python/pythonrun.c:1697
#12 0x000cf723 in PyRun_InteractiveLoopFlags (fp=0xa02e15e0,
filename=0x123994 "<stdin>", flags=0xbffff77c) at Python/pythonrun.c:993
#13 0x000cfa89 in PyRun_AnyFileExFlags (fp=0xa02e15e0, filename=0x123994
"<stdin>", closeit=0, flags=0xbffff77c) at Python/pythonrun.c:962
#14 0x000e47d0 in Py_Main (argc=1, argv=0x2015b0) at Modules/main.c:625
#15 0x00002599 in main (argc=1, argv=0xbffff8cc) at python.c:152
msg89858 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-06-29 16:17
It's a bug in unicode_encode_ucs1: the result string is resized, so 
   str = PyBytes_AS_STRING(res) + (someOffset)
is not valid anymore.

Attached patch+test.
msg89859 - (view) Author: Jerry Chen (jcsalterego) Date: 2009-06-29 16:23
Confirmed patch and test are working
msg89883 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-06-29 21:37
The patch looks good.
msg89889 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-06-29 22:39
Fixed with r73698 (py3k) and r73699.
History
Date User Action Args
2022-04-11 14:56:50adminsetgithub: 50622
2009-06-29 22:39:19amaury.forgeotdarcsetstatus: open -> closed
resolution: fixed
messages: + msg89889
2009-06-29 21:37:35benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg89883
2009-06-29 16:23:23jcsalteregosetmessages: + msg89859
2009-06-29 16:17:15amaury.forgeotdarcsetfiles: + issue6373.patch

nosy: + amaury.forgeotdarc
messages: + msg89858

keywords: + patch
2009-06-29 15:58:51r.david.murraysetpriority: normal
nosy: + loewis
2009-06-29 15:45:44jcsalteregosetnosy: + jcsalterego
messages: + msg89853
2009-06-29 15:31:23glynnccreate