classification
Title: deque pickling problems
Type: Stage:
Components: Extension Modules Versions:
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: Nosy List: ddorfman, rhettinger
Priority: normal Keywords: patch

Created on 2004-11-08 08:10 by ddorfman, last changed 2004-11-09 07:28 by rhettinger. This issue is now closed.

Files
File name Uploaded Description Edit
deque.diff ddorfman, 2004-11-08 08:10 Fix and tests
Messages (5)
msg47270 - (view) Author: Dima Dorfman (ddorfman) Date: 2004-11-08 08:10
Two problems with deque.__reduce__:

  1. If one of the elements is self, it ends up in args,
     which makes it impossible for the unpickler to
     correctly supply it. Fixed by using the listitems
     argument (PEP 307 says that non-lists may use this if
     they support append and extend, which deque does)
     (test_pickle_recursive). This is related to SF #1062277
     but does not depend on it.

  2. Since we wrote our reduce, the standard ones don't have
     a chance to save the instance dictionary, so we have to
     do it ourselves (TestSubclass.test_pickle).
msg47271 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2004-11-08 08:44
Logged In: YES 
user_id=80475

I'll look at this tomorrow.  In the meantime, please check
to see if you have the same problems with set/frozenset.
msg47272 - (view) Author: Dima Dorfman (ddorfman) Date: 2004-11-08 08:51
Logged In: YES 
user_id=908995

I'm working on set. It definitely has this problem, but an
analogous fix requires introducing frozenset.__setstate__
which would make it possible to mutate a frozenset. A more
correct solution might depend on whether someone finds a
better alternative to my patch in #1062277. I think I'll
post a patch that implements frozenset.__setstate__ tonight,
but it's less obviously desirable than this one.
msg47273 - (view) Author: Dima Dorfman (ddorfman) Date: 2004-11-08 11:08
Logged In: YES 
user_id=908995

set patch filed as SF #1062353

I just realized that neither of these patches do anything to
handle __slots__, but I'm not even sure if it's necessary.
At least _reduce_ex tells such classes to write their own
getstate, and such subclasses of set or deque can always
write their own reduce, so lack of such support doesn't make
anything impossible--it just might require a little more
work on the user's part.
msg47274 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2004-11-09 07:28
Logged In: YES 
user_id=80475

Accepted and applied with minor modifications.  See
Modules/collectionsmodule.c 1.36

History
Date User Action Args
2004-11-08 08:10:51ddorfmancreate