classification
Title: array.arrays are not unpickleable
Type: Stage:
Components: Extension Modules Versions: Python 2.4
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: georg.brandl, josiahcarlson, mbrighton, rhettinger, sjmachin, terry.reedy
Priority: high Keywords:

Created on 2005-09-03 20:16 by georg.brandl, last changed 2006-01-10 01:11 by mbrighton. This issue is now closed.

Messages (9)
msg26191 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2005-09-03 20:16
Credits to John Machin for discovering this.

"""
Googling for "pickle array" in comp.lang.python yields
old messages that 
show a PickleError -- plus one message where Alex
Martelli writes "I am 
but an egg" :O)
Looks like arrays are NOW (2.4.1) pickleable but not
unpickleable -- see 
below.
I appreciate that arrays are inherently not pickleable
because of the 
type code.
However:
(1) Anyone know why/when the world changed?
(2) If we had alternative constructors like
array.iarray(contents) in 
parallel to array.array('i', contents), those objects
could be 
pickled/unpickled -- yes/no?

Cheers,
John
====================
Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310
32 bit (Intel)] on 
win32
Type "help", "copyright", "credits" or "license" for
more information.
 >>> import pickle, array
 >>> class Foo(object):
...    pass
...
 >>> foo = Foo()
 >>> foo.ia = array.array('i', [3,2,1])
 >>> foo.ia
array('i', [3, 2, 1])
 >>> s = pickle.dumps(foo, -1)
 >>> bar = pickle.loads(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\Python24\lib\pickle.py", line 1394, in loads
    return Unpickler(file).load()
  File "C:\Python24\lib\pickle.py", line 872, in load
    dispatch[key](self)
  File "C:\Python24\lib\pickle.py", line 1097, in
load_newobj
    obj = cls.__new__(cls, *args)
TypeError: array() takes at least 1 argument (0 given)
===========
"""
msg26192 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2005-09-03 23:20
Logged In: YES 
user_id=80475

In Py2.4, array's became copyable, weak-referencable, and
got support for iterator arguments.  Real pickle support
wasn't added until Py2.5.  The above code fragment is a
by-product of pickle making an incorrect guess at how to
pickle arrays before real pickel support was added.  It is
not really a bug; rather, it begs for a feature that wasn't
added to later.

If it weren't a new feature, I would just backport the 2.5
pickle support. 
msg26193 - (view) Author: John Machin (sjmachin) Date: 2005-09-04 10:41
Logged In: YES 
user_id=480138

Please fix the bug in Python 2.4: if array objects are not
pickleable in 2.4, then pickle and cPickle should raise a
PickleError [like they used to in earlier versions] --
instead of guessing wrongly and misleading callers into
thinking that the objects can be pickled.
msg26194 - (view) Author: Josiah Carlson (josiahcarlson) * Date: 2005-09-08 06:04
Logged In: YES 
user_id=341410

Raymond, they seem to be asking for Pickle and cPickle to
raise an exception when someone attempts to pickle arrays in
a future Python 2.4.2 release.  I don't think that is a new
feature.

As for 2.5, real pickle support seems reasonable.
msg26195 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2005-09-08 06:36
Logged In: YES 
user_id=80475

I think you're misunderstanding.  Direct pickling of arrays
does raise a TypeError.  It would be nice if it also did as
an object attribute; however, I'm not bothered by it enough
to spend development time tracing down the issue and then
altering otherwise correct Py2.4 code just to generate a
prettier message.  It is enough for me that the docs do not
promise pickling, that a message is generated by a direct
attempt to pickle, that the OP's buggy code eventually
errors out, and that everything works fine in Py2.5.

I have no objections to someone finding a way to generate a
better error message but think the time would better be
spent elsewhere.

>>> from array import array
>>> from pickle import dumps, loads
>>> ia = array('i', [3,2,1])
>>> ib = loads(dumps(ia))

Traceback (most recent call last):
 . . .
TypeError: can't pickle array objects
msg26196 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2005-09-08 19:39
Logged In: YES 
user_id=593130

http://python.org/sf/1281556 appears to be a duplicate.
You wish to close it too? (I won't, don't know enough here.)
msg26197 - (view) Author: Mark C. Brighton (mbrighton) Date: 2006-01-09 23:07
Logged In: YES 
user_id=1423362

From the comments, it appears that the developers didn't
notice that the example used protocol '-1' (highest
protocol, which is 2 in Py2.4).  Using protocol 0 (default)
or 1 will give the TypeError (which is appropriate).  

Using protocol 2, pickle.dumps() will accept a pickle object
without an exception, but it doesn't seem to pickle it
properly (see log below).  

Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more
information.
>>> import array
>>> import pickle
>>> a=array.array('B')
>>> s= pickle.dumps(a,2);s
'\x80\x02carray\narray\nq\x00)\x81q\x01.'
>>> pickle.load(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\Python24\lib\pickle.py", line 1390, in load
    return Unpickler(file).load()
  File "C:\Python24\lib\pickle.py", line 855, in __init__
    self.readline = file.readline
AttributeError: 'str' object has no attribute 'readline'
>>>
>>> a=array.array('B', [1,2,3,4,5])
>>> a
array('B', [1, 2, 3, 4, 5])
>>> s2=pickle.dumps(a,2);s2
'\x80\x02carray\narray\nq\x00)\x81q\x01.'
msg26198 - (view) Author: Josiah Carlson (josiahcarlson) * Date: 2006-01-10 01:03
Logged In: YES 
user_id=341410

mbrighton:

You get a string from pickle.dumps(), and to load strings,
you should use pickle.loads() .

The AttributeError you are recieving is the result of using
pickle.load() with a string, not a file object, as required,
and as described by the documentation.
msg26199 - (view) Author: Mark C. Brighton (mbrighton) Date: 2006-01-10 01:11
Logged In: YES 
user_id=1423362

Thatks for catching that.  Unfortunatly, that's just a
function of me editing the example.  I ment to edit that out
and instead took out the exception I was trying to show. 
Here's what I ment to demonstrate:

>>> import array
>>> import pickle
>>> a=array.array('B')
>>> s= pickle.dumps(a,2);s
'\x80\x02carray\narray\nq\x00)\x81q\x01.'
>>> pickle.loads(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\Python24\lib\pickle.py", line 1394, in loads
    return Unpickler(file).load()
  File "C:\Python24\lib\pickle.py", line 872, in load
    dispatch[key](self)
  File "C:\Python24\lib\pickle.py", line 1097, in load_newobj
    obj = cls.__new__(cls, *args)
TypeError: array() takes at least 1 argument (0 given)

The part of the problem appears to be with dumps(), as the
same string is returned independent of the array provided:

>>> pickle.dumps(array.array('B'),2)
'\x80\x02carray\narray\nq\x00)\x81q\x01.'
>>> pickle.dumps(array.array('B', [1,2,3,4,5]),2)
'\x80\x02carray\narray\nq\x00)\x81q\x01.'
>>> pickle.dumps(array.array('B', [230,]*100),2)
'\x80\x02carray\narray\nq\x00)\x81q\x01.'
>>> pickle.dumps(array.array('H', [1,2,3,4,5]),2)
'\x80\x02carray\narray\nq\x00)\x81q\x01.'
History
Date User Action Args
2005-09-03 20:16:53birkenfeldcreate