classification
Title: cPickle gives strange error for large objects.
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: alexandre.vassalotti, amaury.forgeotdarc, meawoppl, santoso.wijaya, serhiy.storchaka, xuanji
Priority: normal Keywords:

Created on 2011-04-19 08:06 by meawoppl, last changed 2015-11-26 17:37 by serhiy.storchaka. This issue is now closed.

Messages (4)
msg134017 - (view) Author: MattyG (meawoppl) Date: 2011-04-19 08:06
from numpy import *
import cPickle

a = zeros((300000, 1000)) 
f = open("test.pkl", "w") 
cPickle.dump(a, f)

------
SystemError: Traceback (most recent call last)

/home/kddcup/code/matt/svd-projection/take5/<ipython console> in <module>()

SystemError?: error return without exception set

Or using the .dump function:

a.dump("test.pkl")

SystemError? Traceback (most recent call last)

/home/kddcup/code/matt/svd-projection/take5/<ipython console> in <module>()

SystemError: NULL result without error in PyObject?_Call

I am not sure if this is a numpy or Pickle/cPickle glitch. In either case, even if this is a semi-known behavior, a more instructive error message would certainly help. I think the problem only happens for arrays larger than 2**(32-1) bytes but I would have to experiment more to be sure.
msg134020 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2011-04-19 08:19
See also issue11564 and issue9614.
msg188284 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2013-05-02 22:43
It is an integer overflow issue. It is easy to reproduce without numpy:

$ python2.7 -c "import cPickle; cPickle.dumps('\x00' * 2**31)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
SystemError: error return without exception set

We fixed this in Python 3. It may be worthwhile to fix cPickle as well.
msg247317 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-07-25 06:36
Is this issue still actual?
History
Date User Action Args
2015-11-26 17:37:34serhiy.storchakasetstatus: pending -> closed
resolution: out of date
stage: needs patch -> resolved
2015-07-25 06:36:51serhiy.storchakasetstatus: open -> pending
nosy: + serhiy.storchaka
messages: + msg247317

2013-05-02 22:43:31alexandre.vassalottisetnosy: + alexandre.vassalotti

messages: + msg188284
stage: needs patch
2011-04-26 17:40:35santoso.wijayasetnosy: + santoso.wijaya
2011-04-26 15:02:01xuanjisetnosy: + xuanji
2011-04-19 08:19:54amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg134020
2011-04-19 08:06:00meawopplcreate