classification
Title: pickling files works with protocol=2.
Type: behavior Stage: needs patch
Components: Library (Lib) Versions: Python 3.2, Python 3.1, Python 2.7, Python 2.6, Python 2.5, Python 2.4
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: alexandre.vassalotti, kirill_simonov, pitrou, schmir (4)
Priority: normal Keywords patch

Created on 2006-04-22 16:20 by kirill_simonov, last changed 2009-06-29 11:48 by pitrou.

Files
File name Uploaded Description Edit Remove
disable_file_pickling-py2.diff alexandre.vassalotti, 2009-06-28 20:27
Messages (3)
msg60906 - (view) Author: Kirill Simonov (kirill_simonov) Date: 2006-04-22 16:20
Compare:
>>> import pickle
>>> pickle.dumps(file('/etc/passwd'))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.4/pickle.py", line 1386, in dumps
    Pickler(file, protocol, bin).dump(obj)
  File "/usr/lib/python2.4/pickle.py", line 231, in dump
    self.save(obj)
  File "/usr/lib/python2.4/pickle.py", line 313, in save
    rv = reduce(self.proto)
  File "/usr/lib/python2.4/copy_reg.py", line 69, in
_reduce_ex
    raise TypeError, "can't pickle %s objects" %
base.__name__
TypeError: can't pickle file objects

with
>>> pickle.dumps(file('/etc/passwd'), protocol=2)
'\x80\x02c__builtin__\nfile\nq\x00)\x81q\x01.'

Similarly, the __reduce__ method works for basic
objects like str, int or dict with protocol=2, but
doesn't work with protocol=1:

>>> (1).__reduce_ex__(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.4/copy_reg.py", line 69, in
_reduce_ex
    raise TypeError, "can't pickle %s objects" %
base.__name__
TypeError: can't pickle int objects
>>> (1).__reduce_ex__(2)
(<function __newobj__ at 0xb7e5117c>, (<type 'int'>,
1), None, None, None)
msg89783 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) Date: 2009-06-28 20:27
That seems easy to fix by adding a dummy __reduce__ method to file. My
only worry is this could break file subclasses which may have ad-hoc
mechanisms implemented for pickling files.
msg89829 - (view) Author: Antoine Pitrou (pitrou) Date: 2009-06-29 11:48
Pickling a file is tricky. You can't just pickle the construction
parameters, you also need to save (and restore) the whole buffering
state. I'm not sure a half-working pickle solution would make a good
service to the user.
History
Date User Action Args
2009-06-29 11:48:33pitrousetnosy: + pitrou
messages: + msg89829
2009-06-28 20:27:56alexandre.vassalottisetfiles: + disable_file_pickling-py2.diff

nosy: + alexandre.vassalotti
versions: + Python 2.6, Python 3.1, Python 2.7, Python 3.2
messages: + msg89783

keywords: + patch
type: behavior
stage: needs patch
2008-03-14 17:59:00schmirsetnosy: + schmir
versions: + Python 2.5
2006-04-22 16:20:51kirill_simonovcreate