Description
===========
Running the attached example program with Python 2.7.12 produces the output below. The demo deliberately raises a user-defined exception during the unpickling process, but the problem is that this exception does not propagate out of the unpickle call. Instead, it gets converted into a TypeError which is confusing and does not help identify the original problem.
INFO:root:Creating BadReduce object
INFO:root:Pickling
INFO:root:Unpickling
INFO:root:Raising exception "BadReduce init failed"
Traceback (most recent call last):
File "cpickle_reduce_failure.py", line 48, in <module>
main()
File "cpickle_reduce_failure.py", line 41, in main
pickler.loads(s1)
File "cpickle_reduce_failure.py", line 27, in __init__
raise exception
TypeError: __init__() takes exactly 2 arguments (4 given)
If you change the demo to use the Python pickle module, i.e. "import pickle as pickler", it produces the expected output below:
INFO:root:Creating BadReduce object
INFO:root:Pickling
INFO:root:Unpickling
INFO:root:Raising exception "BadReduce init failed"
INFO:root:Got MyException "BadReduce init failed"
INFO:root:Done
Analysis
========
The following code in Modules/cPickle.c in the function Instance_New (around https://github.com/python/cpython/blob/2.7/Modules/cPickle.c#L3917) does a PyErr_Restore with the exception type MyException, as raised from BadReduce.__init__, but it replaces the exception value with a tuple (original_exception, cls, args):
PyObject *tp, *v, *tb, *tmp_value;
PyErr_Fetch(&tp, &v, &tb);
tmp_value = v;
/* NULL occurs when there was a KeyboardInterrupt */
if (tmp_value == NULL)
tmp_value = Py_None;
if ((r = PyTuple_Pack(3, tmp_value, cls, args))) {
Py_XDECREF(v);
v=r;
}
PyErr_Restore(tp,v,tb);
Later, PyErr_NormalizeException attempts to convert the exception value (the tuple) to the original exception type. This fails because MyException.__init__() can't handle the multiple arguments. This is what produces the TypeError "__init__() takes exactly 2 arguments (4 given)"
Proposed Fix
============
I think it would be better if Instance_New did the PyErr_NormalizeException itself instead of allowing it to be done lazily. If the normalize works, i.e. the exception type accepts the extra arguments, the behaviour would be almost unchanged - the only difference being the PyErr_NormalizeException happens earlier. However, if the normalize fails, Instance_New can restore the original exception unchanged. That means that we lose the cls, args information in this case, but not the original exception.
|