classification
Title: pickle segfault with infinite loop in __getattr__
Type: Stage:
Components: Library (Lib) Versions: Python 3.0
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: alexandre.vassalotti, erickt, gvanrossum, idadesub
Priority: normal Keywords:

Created on 2008-08-07 07:41 by erickt, last changed 2008-08-15 03:08 by alexandre.vassalotti. This issue is now closed.

Messages (6)
msg70815 - (view) Author: Erick Tryzelaar (erickt) Date: 2008-08-07 07:41
I found a segfault in pickle.load when you overload __getattr__ and 
create yourself a infinite loop in the latest svn checkout of python 3:

########################################
import pickle

class Foo:
    def __getattr__(self, key):
        self.foo

with open('foo.db', 'wb') as f:
    foo = Foo()
    pickle.dump(foo, f)

with open('foo.db', 'rb') as f:
    pickle.load(f)
########################################

This results in this stack trace on my mac:

Reason: KERN_PROTECTION_FAILURE at address: 0x0000000c
0x0000dc6b in PyObject_Call (func=0x0, arg=0x44cd58, kw=0x0) at 
Objects/abstract.c:2174
2174		if ((call = func->ob_type->tp_call) != NULL) {
(gdb) bt
#0  0x0000dc6b in PyObject_Call (func=0x0, arg=0x44cd58, kw=0x0) at 
Objects/abstract.c:2174
#1  0x004c1b4d in unpickler_call (self=0x4a6240, func=0x0, arg=0x4b66c8) 
at /Users/Shared/erickt/Projects/py3k.svn/Modules/_pickle.c:413
#2  0x004cac9a in load_build (self=0x4a6240) at 
/Users/Shared/erickt/Projects/py3k.svn/Modules/_pickle.c:3844
#3  0x004cbb4f in load (self=0x4a6240) at 
/Users/Shared/erickt/Projects/py3k.svn/Modules/_pickle.c:4047
#4  0x004cbe71 in Unpickler_load (self=0x4a6240) at 
/Users/Shared/erickt/Projects/py3k.svn/Modules/_pickle.c:4119
#5  0x000f2fef in call_function (pp_stack=0xbfffea84, oparg=0) at 
Python/ceval.c:3387
#6  0x000edfdb in PyEval_EvalFrameEx (f=0x326cd8, throwflag=0) at 
Python/ceval.c:2205
#7  0x000f157e in PyEval_EvalCodeEx (co=0x4a9628, globals=0x487f50, 
locals=0x0, args=0x32593c, argcount=1, kws=0x325940, kwcount=0, 
defs=0x0, defcount=0, kwdefs=0x4b6428, closure=0x0) at 
Python/ceval.c:2840
#8  0x000f39e5 in fast_function (func=0x4b4ab8, pp_stack=0xbfffee54, 
n=1, na=1, nk=0) at Python/ceval.c:3501
#9  0x000f35cf in call_function (pp_stack=0xbfffee54, oparg=1) at 
Python/ceval.c:3424
#10 0x000edfdb in PyEval_EvalFrameEx (f=0x3257f8, throwflag=0) at 
Python/ceval.c:2205
#11 0x000f157e in PyEval_EvalCodeEx (co=0x444c28, globals=0x255818, 
locals=0x255818, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, 
defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:2840
#12 0x000e564f in PyEval_EvalCode (co=0x444c28, globals=0x255818, 
locals=0x255818) at Python/ceval.c:519
#13 0x00122a96 in run_mod (mod=0x872c80, filename=0xbffff228 "foo.py", 
globals=0x255818, locals=0x255818, flags=0xbffff628, arena=0x322020) at 
Python/pythonrun.c:1553
#14 0x00122884 in PyRun_FileExFlags (fp=0xa00dcde0, filename=0xbffff228 
"foo.py", start=257, globals=0x255818, locals=0x255818, closeit=1, 
flags=0xbffff628) at Python/pythonrun.c:1510
#15 0x00120e39 in PyRun_SimpleFileExFlags (fp=0xa00dcde0, 
filename=0xbffff228 "foo.py", closeit=1, flags=0xbffff628) at 
Python/pythonrun.c:1048
#16 0x001202f9 in PyRun_AnyFileExFlags (fp=0xa00dcde0, 
filename=0xbffff228 "foo.py", closeit=1, flags=0xbffff628) at 
Python/pythonrun.c:845
#17 0x00134d1c in Py_Main (argc=2, argv=0x227028) at Modules/main.c:592
#18 0x00002574 in main (argc=2, argv=0xbffff748) at python.c:57


It seems that this isn't just for infinite loops. If you replace the 
class with this:


class Foo:
    def __init__(self):
        self.foo = {}

    def __getattr__(self, key):
        self.foo[5]


It still errors out. So I'm guessing pickle is just not handling 
exceptions properly.
msg70836 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2008-08-07 16:30
Does this occur in 2.6 or 2.5?
msg70837 - (view) Author: Erick Tryzelaar (idadesub) Date: 2008-08-07 16:39
> Guido van Rossum <guido@python.org> added the comment:
>
> Does this occur in 2.6 or 2.5?

It doesn't in python 2.5. The RuntimeError manages to get printed out.
I don't have 2.6 installed to test against at the moment. Here's the
equivalent code:

########################
import pickle

class Foo:
    def __getattr__(self, key):
        self.foo

f = open('foo.db', 'w')
foo = Foo()
pickle.dump(foo, f)
f.close()

f = open('foo.db', 'r')
pickle.load(f)
f.close()
########################

This also happens with cPickle.
msg71001 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2008-08-11 05:48
This is a bug in the C implementation of pickle (i.e., the _pickle
module). I think you're right about the missing exception check. At
first glance, it looks like the missing else-if case for "setstate ==
NULL", in load_build(), is the cause of the problem:

static int
load_build(UnpicklerObject *self)
{
...
    setstate = PyObject_GetAttrString(inst, "__setstate__");
    if (setstate == NULL && PyErr_ExceptionMatches(PyExc_AttributeError)) {
        PyErr_Clear();
    }
/*---missing else-if case---------
    else if (setstate == NULL) {
        return NULL;
    }
----------------------------------*/
    else {
        PyObject *result;

        /* The explicit __setstate__ is responsible for everything. */
        result = unpickler_call(self, setstate, state);
        Py_DECREF(setstate);
        if (result == NULL)
            return -1;
        Py_DECREF(result);
        return 0;
    }
...
msg71036 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2008-08-11 23:39
Good find Alexandre!  Can you work up a patch?  Would be even better if
it had a unittest.
msg71158 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2008-08-15 03:08
Committed fix in r65689. Thanks!
History
Date User Action Args
2008-08-15 03:08:21alexandre.vassalottisetstatus: open -> closed
resolution: fixed
messages: + msg71158
2008-08-11 23:39:21gvanrossumsetmessages: + msg71036
2008-08-11 05:48:47alexandre.vassalottisetpriority: normal
nosy: + alexandre.vassalotti
messages: + msg71001
2008-08-07 16:39:09idadesubsetnosy: + idadesub
messages: + msg70837
2008-08-07 16:30:03gvanrossumsetnosy: + gvanrossum
messages: + msg70836
2008-08-07 07:41:23ericktcreate