classification
Title: cPickle is seriously broken
Type: behavior Stage:
Components: Versions: Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: facundobatista Nosy List: cuerty, facundobatista, jcea, schmir
Priority: normal Keywords: patch

Created on 2008-06-23 14:42 by schmir, last changed 2008-06-26 07:21 by jcea. This issue is now closed.

Files
File name Uploaded Description Edit
test-3179.diff schmir, 2008-06-23 15:11 test case
Messages (8)
msg68629 - (view) Author: Ralf Schmitt (schmir) Date: 2008-06-23 14:42
import cPickle

res=[]
for x in range(1,2000):
    res.append(dict(doc=x, similar=[]))

cPickle.dumps(res)


Traceback (most recent call last):
  File "pi.py", line 10, in <module>
cPickle.dumps(res)
RuntimeError: maximum recursion depth exceeded

svn r64471 seems to cause the problematic behaviour.

facundo, you committed that one.
msg68631 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2008-06-23 14:53
This example works before my patch, exactly!

The reason of the patch I applied is that in some cases a Recursion
error should be raised, but it didn't happen, causing some serious
issues later.

I'm putting in copy to cuerty, for him to analyze if this case should
have caused Recursion error in the first place, or not.

In any case, it's a good example. Ralf, would you mind to generate a
test case from this example?

Thank you all!
msg68632 - (view) Author: Ralf Schmitt (schmir) Date: 2008-06-23 14:56
Apparently there are some self->nesting-- calls missing in batch_list
and batch_dict.
msg68633 - (view) Author: Ralf Schmitt (schmir) Date: 2008-06-23 14:57
Of course it should not raise an RecursionError.

for reference:
http://bugs.python.org/issue2702 is the original bugreport.
msg68635 - (view) Author: Ralf Schmitt (schmir) Date: 2008-06-23 15:11
here is a test case. I cannot run it however:
~/pydev/trunk/ ./python Lib/test/test_cpickle.py                   
ralf@red ok
Traceback (most recent call last):
  File "Lib/test/test_cpickle.py", line 3, in <module>
from test.pickletester import AbstractPickleTests, AbstractPickleModuleTests
ImportError: No module named pickletester

???
msg68669 - (view) Author: Ralf Schmitt (schmir) Date: 2008-06-24 06:52
the test works as expected (i.e. it fails). The problem I had was that 
some Bittorrent bencode also installed a test package for me.


======================================================================
ERROR: test_flat_list (__main__.cPickleFlatList)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_cpickle.py", line 113, in test_flat_list
    cPickle.dumps(lst)
RuntimeError: maximum recursion depth exceeded

----------------------------------------------------------------------
Ran 159 tests in 0.457s

FAILED (errors=1)
Traceback (most recent call last):
  File "Lib/test/test_cpickle.py", line 128, in <module>
test_main()
  File "Lib/test/test_cpickle.py", line 124, in test_main
cPickleFlatList,
  File "/home/ralf/pydev/trunk/Lib/test/test_support.py", line 714, in
run_unittest
_run_suite(suite)
  File "/home/ralf/pydev/trunk/Lib/test/test_support.py", line 697, in
_run_suite
raise TestFailed(err)
test.test_support.TestFailed: Traceback (most recent call last):
  File "Lib/test/test_cpickle.py", line 113, in test_flat_list
    cPickle.dumps(lst)
RuntimeError: maximum recursion depth exceeded
msg68670 - (view) Author: Ralf Schmitt (schmir) Date: 2008-06-24 06:53
btw. this should be a release blocker.
msg68746 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2008-06-25 19:27
I reverted the patch, commented out the previous test, and included this
one in the test suite, to never break it again, ;)

I hope we now find a solution to the issue #2702.

Thank you!!
History
Date User Action Args
2008-06-26 07:21:01jceasetnosy: + jcea
2008-06-25 19:28:08facundobatistasetstatus: open -> closed
resolution: fixed
2008-06-25 19:27:57facundobatistasetmessages: + msg68746
2008-06-24 06:53:05schmirsetmessages: + msg68670
2008-06-24 06:52:33schmirsetmessages: + msg68669
2008-06-23 23:08:26facundobatistasetassignee: facundobatista
2008-06-23 15:11:08schmirsetfiles: + test-3179.diff
keywords: + patch
messages: + msg68635
2008-06-23 14:57:43schmirsetmessages: + msg68633
2008-06-23 14:56:16schmirsetmessages: + msg68632
2008-06-23 14:53:50facundobatistasetnosy: + cuerty
messages: + msg68631
2008-06-23 14:42:12schmircreate