classification
Title: pickling files works with protocol=2.
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: alexandre.vassalotti, eric.araujo, kirill_simonov, pitrou, schmir
Priority: normal Keywords: patch

Created on 2006-04-22 16:20 by kirill_simonov, last changed 2010-08-05 07:20 by alexandre.vassalotti. This issue is now closed.

Files
File name Uploaded Description Edit
disable_file_pickling-py2.diff alexandre.vassalotti, 2009-06-28 20:27
Messages (5)
msg60906 - (view) Author: Kirill Simonov (kirill_simonov) Date: 2006-04-22 16:20
Compare:
>>> import pickle
>>> pickle.dumps(file('/etc/passwd'))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.4/pickle.py", line 1386, in dumps
    Pickler(file, protocol, bin).dump(obj)
  File "/usr/lib/python2.4/pickle.py", line 231, in dump
    self.save(obj)
  File "/usr/lib/python2.4/pickle.py", line 313, in save
    rv = reduce(self.proto)
  File "/usr/lib/python2.4/copy_reg.py", line 69, in
_reduce_ex
    raise TypeError, "can't pickle %s objects" %
base.__name__
TypeError: can't pickle file objects

with
>>> pickle.dumps(file('/etc/passwd'), protocol=2)
'\x80\x02c__builtin__\nfile\nq\x00)\x81q\x01.'

Similarly, the __reduce__ method works for basic
objects like str, int or dict with protocol=2, but
doesn't work with protocol=1:

>>> (1).__reduce_ex__(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.4/copy_reg.py", line 69, in
_reduce_ex
    raise TypeError, "can't pickle %s objects" %
base.__name__
TypeError: can't pickle int objects
>>> (1).__reduce_ex__(2)
(<function __newobj__ at 0xb7e5117c>, (<type 'int'>,
1), None, None, None)
msg89783 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2009-06-28 20:27
That seems easy to fix by adding a dummy __reduce__ method to file. My
only worry is this could break file subclasses which may have ad-hoc
mechanisms implemented for pickling files.
msg89829 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-06-29 11:48
Pickling a file is tricky. You can't just pickle the construction
parameters, you also need to save (and restore) the whole buffering
state. I'm not sure a half-working pickle solution would make a good
service to the user.
msg105404 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-05-09 17:36
Antoine, I think the goal here is to disable pickling, not making it work.

Alexandre, is it ok for a class to define pickle magic methods just to raise an exception? Doesn’t it break expectations?

One question hasn’t been answered: When did the protocol change? Was it intentional or is it a bug?
msg112951 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2010-08-05 07:20
Although it is tempting to fix this, it feels to much like a feature. I am closing as won't fix because we don't add new features to 2.x.
History
Date User Action Args
2010-08-05 07:20:40alexandre.vassalottisetstatus: open -> closed
versions: - Python 3.1, Python 3.2
messages: + msg112951

resolution: wont fix
stage: needs patch -> resolved
2010-08-05 00:05:43terry.reedysetversions: - Python 2.4
2010-08-05 00:05:29terry.reedysetversions: - Python 2.6, Python 2.5
2010-05-09 17:36:14eric.araujosetnosy: + eric.araujo
messages: + msg105404
2009-06-29 11:48:33pitrousetnosy: + pitrou
messages: + msg89829
2009-06-28 20:27:56alexandre.vassalottisetfiles: + disable_file_pickling-py2.diff

type: behavior
versions: + Python 2.6, Python 3.1, Python 2.7, Python 3.2
keywords: + patch
nosy: + alexandre.vassalotti

messages: + msg89783
stage: needs patch
2008-03-14 17:59:00schmirsetnosy: + schmir
versions: + Python 2.5
2006-04-22 16:20:51kirill_simonovcreate