classification
Title: Pickling bound methods
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions: Python 3.2
process
Status: closed Resolution: duplicate
Dependencies: Superseder: pickle should support methods
View: 9276
Assigned To: Nosy List: alexandre.vassalotti, belopolsky, hinsen, loewis, obamausa8, pitrou, rhettinger
Priority: normal Keywords:

Created on 2002-05-20 11:17 by hinsen, last changed 2010-08-02 15:19 by pitrou. This issue is now closed.

Messages (14)
msg53542 - (view) Author: Konrad Hinsen (hinsen) Date: 2002-05-20 11:17
Last week I noticed that the pickle and cPickle modules
cannot handle bound methods. I found a solution that is
simple and (I think) general, so perhaps it should
become part of the standard library.

Here is my code:

import copy_reg

def pickle_bound_method(method):
    return getattr, (method.im_self, method.__name__)

class _Foo:
    def bar(self):
        pass

_foo = _Foo()

copy_reg.constructor(getattr)
copy_reg.pickle(type(_foo.bar), pickle_bound_method)

msg53543 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2002-06-09 17:14
Logged In: YES 
user_id=21627

Making getattr a safe_constructor has security implictions
which make this approach dangerous. It seems that unpickling
might invoke arbitrary __getattr__ implementations. Adding a
protocol to declare classes as "safe for getattr" might help.
msg53544 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2002-09-22 13:08
Logged In: YES 
user_id=21627

Can you perhaps rewrite this to use new.instancemethod?
msg53545 - (view) Author: Konrad Hinsen (hinsen) Date: 2002-09-24 17:54
Logged In: YES 
user_id=11850

Of course:

import copy_reg

def get_method(object, name):
    klass = object.__class__
    fn = klass.__dict__[name]
    return new.instancemethod(fn, object, klass)

def pickle_bound_method(method):
    return get_method, (method.im_self, method.__name__)

class _Foo:
    def bar(self):
        pass

_foo = _Foo()

copy_reg.constructor(get_method)
copy_reg.pickle(type(_foo.bar), pickle_bound_method)
msg53546 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2002-09-24 18:12
Logged In: YES 
user_id=21627

That sounds good to me; I'm going to install it.
msg53547 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2002-09-30 10:03
Logged In: YES 
user_id=21627

Reconsidering: It fails to work in the presence of
inheritance, right?

>>> class B:
...   def foo(self):pass
... 
>>> class D(B):pass
... 
>>> d = D()
>>> m = d.foo
>>> m.im_self
<__main__.D instance at 0x1ccb28>
>>> m.__name__
'foo'
>>> D.__dict__['foo']
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
KeyError: foo
>>> 
msg53548 - (view) Author: Konrad Hinsen (hinsen) Date: 2002-09-30 13:36
Logged In: YES 
user_id=11850

True. Would it be a security problem to use getattr inside a 
specialized constructor? For example,

def get_method(object, name):
    klass = object.__class__
    fn = getattr(klass, name)
    return new.instancemethod(fn, object, klass)

This would be difficult to abuse as a general getattr 
replacement, because new.instancemethod would fail in most 
cases other than the intended one.
msg53549 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2002-09-30 15:35
Logged In: YES 
user_id=21627

My concern is that the getattr call already creates a
danger, since you just have to find a 'suitable' object
whose __getattr__ does funny things. In the revised form,
this object would also have to appear as the __class__ of
some other object, so I guess this is pretty safe. I'll ask
python-dev.
msg53550 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2004-01-01 07:17
Logged In: YES 
user_id=80475

Any progress on this one?
msg53551 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2004-01-01 12:48
Logged In: YES 
user_id=21627

There was some discussion on python-dev (initiated by
Christian) on whether bound methods should provide an
__reduce__. Apart from that, no progress.
msg65867 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2008-04-27 01:58
Personally, I don't see how adding this feature would create a security
hole (or more properly said, grow the giant hole that are the GLOBAL and
 REDUCE opcodes). I don't see either why this should be included in the
standard library.

Unless there is still a need for this feature, I think this bug should
be closed.
msg89816 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2009-06-29 06:42
I am leaving this issue open for now.

I reconsidered whether we should add pickle support for methods and I
now think it would probably be a good idea. For example, the
multiprocessing module would benefit from a having built-in support for
method pickling (right now, it has to subclass Pickler to pickle methods).
msg110474 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-07-16 19:26
See also issue3657. For some reasons, C bound methods aliased as module globals can be pickled, but not similarly aliased Python bound methods:

>>> random.random, random.seed
(<built-in method random of Random object at 0x27abe20>, <bound method Random.seed of <random.Random object at 0x27abe20>>)
>>> pickle.dumps(random.random)
b'\x80\x03crandom\nrandom\nq\x00.'
>>> pickle.dumps(random.seed)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/antoine/py3k/__svn__/Lib/pickle.py", line 1314, in dumps
    Pickler(f, protocol, fix_imports=fix_imports).dump(obj)
_pickle.PicklingError: Can't pickle <class 'method'>: attribute lookup builtins.method failed
msg112487 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-08-02 15:19
Let's continue the discussion on issue9276.
History
Date User Action Args
2010-08-02 15:19:19pitrousetstatus: open -> closed

nosy: - eric.araujo
messages: + msg112487

superseder: pickle should support methods
resolution: duplicate
2010-08-02 14:57:26eric.araujosetnosy: + eric.araujo
2010-07-16 19:26:51pitrousetversions: + Python 3.2
nosy: + pitrou, belopolsky

messages: + msg110474

assignee: loewis ->
stage: needs patch
2009-06-29 07:10:10alexandre.vassalottisetmessages: - msg89818
2009-06-29 06:57:48obamausa8setnosy: + obamausa8
messages: + msg89818
2009-06-29 06:42:16alexandre.vassalottisetstatus: pending -> open

messages: + msg89816
2008-04-27 02:12:01alexandre.vassalottisetstatus: open -> pending
2008-04-27 01:58:48alexandre.vassalottisetnosy: + alexandre.vassalotti
messages: + msg65867
2002-05-20 11:17:41hinsencreate