classification
Title: pickle/cPickle of recursive tuples create pickles that cPickle can't load
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.1, Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: collinwinter Nosy List: alexandre.vassalotti, collinwinter, cwitty
Priority: normal Keywords: 26backport, easy, patch

Created on 2009-04-19 19:52 by cwitty, last changed 2009-05-26 16:55 by collinwinter. This issue is now closed.

Files
File name Uploaded Description Edit
rec_tuple.patch collinwinter, 2009-05-25 02:09 Patch against trunk, r72906
Messages (5)
msg86177 - (view) Author: Carl Witty (cwitty) Date: 2009-04-19 19:52
When I try to pickle a recursive tuple (that recurses through a list),
pickle can load the result but cPickle fails with "unpickling stack
overflow".

(I just downloaded and built Python 2.6.2 on 64-bit x86 Debian testing
to verify this bug; it also fails on Python 2.5.2 on 32-bit x86 Debian
testing.  I think that probably it doesn't depend on the architecture or
operating system at all.)

Here is the test case.  At the end, cPickle.loads raises an exception;
instead, it should produce an object equivalent to v, like pickle.loads
does.  (I didn't show it in the test case, but pickles produced by
pickle.dumps have the same behavior -- they work with pickle.loads but
not cPickle.loads.)

>>> v = ([],)
>>> v[0].append(v)
>>> import pickle
>>> import cPickle
>>> v
([([...],)],)
>>> vp = cPickle.dumps(v)
>>> pickle.loads(vp)
([([...],)],)
>>> cPickle.loads(vp)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
cPickle.UnpicklingError: unpickling stack underflow
msg86416 - (view) Author: Collin Winter (collinwinter) * (Python committer) Date: 2009-04-24 16:51
Interestingly, it only fails with protocol 0:

>>> v = ([],)
>>> v[0].append(v)
>>> import pickle,cPickle
>>> cPickle.loads(pickle.dumps(v, 0))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
cPickle.UnpicklingError: unpickling stack underflow
>>> cPickle.loads(pickle.dumps(v, 1))
([([...],)],)
>>> cPickle.loads(pickle.dumps(v, 2))
([([...],)],)
>>>


I'll see if I can come up with a fix.
msg88300 - (view) Author: Collin Winter (collinwinter) * (Python committer) Date: 2009-05-25 02:09
Bug-fix patch attached. Alexandre, can you take a look? Feel free to
bounce it back if you don't have time.

I'll port to 2.6 and py3k once this is reviewed for trunk.
msg88322 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2009-05-25 19:21
Looks good to me.
msg88376 - (view) Author: Collin Winter (collinwinter) * (Python committer) Date: 2009-05-26 16:55
Fixed in r72930 (trunk), r72931 (2.6), r72942 (py3k).
History
Date User Action Args
2009-05-26 16:55:19collinwintersetstatus: open -> closed
resolution: fixed
messages: + msg88376

stage: patch review -> resolved
2009-05-25 19:21:38alexandre.vassalottisetmessages: + msg88322
2009-05-25 02:09:48collinwintersetfiles: + rec_tuple.patch

versions: + Python 3.1, Python 2.7, - Python 2.5
keywords: + patch, easy, 26backport
nosy: + alexandre.vassalotti

messages: + msg88300
stage: patch review
2009-04-24 16:51:24collinwintersetassignee: collinwinter
messages: + msg86416
2009-04-24 16:14:11collinwintersetnosy: + collinwinter
2009-04-19 19:52:59cwittycreate