classification
Title: Pickling of exceptions broken
Type: Stage:
Components: None Versions: Python 2.5
process
Status: closed Resolution:
Dependencies: Superseder: Fix exception pickling: Move initial args assignment to BaseException.__new__
View: 1692335
Assigned To: Nosy List: dcjim, georg.brandl, zseil
Priority: normal Keywords:

Created on 2007-06-25 14:43 by dcjim, last changed 2007-08-23 17:24 by georg.brandl. This issue is now closed.

Messages (7)
msg32389 - (view) Author: Jim Fulton (dcjim) (Python triager) Date: 2007-06-25 14:43
Exceptions with required initialization arguments can't be unpickled:

>>> class E(Exception):
...     def __init__(self, x):
...         self.x = x
... 
>>> import pickle
>>> e = E(1)
>>> p = pickle.dumps(e, 1)
>>> pickle.loads(p)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/python/2.5.1/lib/python2.5/pickle.py", line 1374, in loads
    return Unpickler(file).load()
  File "/usr/local/python/2.5.1/lib/python2.5/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/local/python/2.5.1/lib/python2.5/pickle.py", line 1133, in load_reduce
    value = func(*args)
TypeError: __init__() takes exactly 2 arguments (1 given)

This is because __reduce__ defined in exceptions.c returns the type and the args variable, which an exception subclass might not populate.

Also, the reduce implementation doesn't properly serialize the message attribute.

I assume that the need for a custom reduce is due to the micro-optimization to store arge and message in C slots.  Is this really necessary?
msg32390 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-06-25 15:26
AFAIR we were told that filling args and message is part of the exception contract...
msg32391 - (view) Author: Jim Fulton (dcjim) (Python triager) Date: 2007-06-25 15:53
I'm not aware of any such contract. Can you point to anything in writing?

See for example:

file:///home/jim/Documentation/Python-Docs-2.4.1/tut/node10.html#SECTION0010500000000000000000

which teaches people to create custom exceptions that:

- don't set args ro message and

- won't be unpicklable in Python 2.5.

Also, as I mentioned, the reduce implementation doesn't preserve the message, so even if that was the contract, the contract is broken.
msg32392 - (view) Author: Jim Fulton (dcjim) (Python triager) Date: 2007-06-25 15:57
I'll note that I think the right thing to do is to:

- Take args and message out of the C struct.

- inherit the default reduce behavior from object.
msg32393 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-06-25 15:57
No, I haven't anything written -- and this should be fixed anyway.
msg32394 - (view) Author: Ziga Seilnacht (zseil) * (Python committer) Date: 2007-08-12 11:26
Jim, could you please take a look at the pathes in #1692335?
See: http://www.python.org/sf/1692335
msg55163 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-08-23 17:24
Closing in favor of #1692335.
History
Date User Action Args
2007-08-23 17:24:47georg.brandlsetstatus: open -> closed
superseder: Fix exception pickling: Move initial args assignment to BaseException.__new__
messages: + msg55163
2007-06-25 14:43:03dcjimcreate