Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pickling bound methods #36622

Closed
hinsen mannequin opened this issue May 20, 2002 · 14 comments
Closed

Pickling bound methods #36622

hinsen mannequin opened this issue May 20, 2002 · 14 comments
Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@hinsen
Copy link
Mannequin

hinsen mannequin commented May 20, 2002

BPO 558238
Nosy @loewis, @rhettinger, @abalkin, @pitrou, @avassalotti
Superseder
  • bpo-9276: pickle should support methods
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2010-08-02.15:19:19.382>
    created_at = <Date 2002-05-20.11:17:41.000>
    labels = ['type-feature', 'library']
    title = 'Pickling bound methods'
    updated_at = <Date 2010-08-02.15:19:19.380>
    user = 'https://bugs.python.org/hinsen'

    bugs.python.org fields:

    activity = <Date 2010-08-02.15:19:19.380>
    actor = 'pitrou'
    assignee = 'none'
    closed = True
    closed_date = <Date 2010-08-02.15:19:19.382>
    closer = 'pitrou'
    components = ['Library (Lib)']
    creation = <Date 2002-05-20.11:17:41.000>
    creator = 'hinsen'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 558238
    keywords = []
    message_count = 14.0
    messages = ['53542', '53543', '53544', '53545', '53546', '53547', '53548', '53549', '53550', '53551', '65867', '89816', '110474', '112487']
    nosy_count = 7.0
    nosy_names = ['loewis', 'rhettinger', 'hinsen', 'belopolsky', 'pitrou', 'alexandre.vassalotti', 'obamausa8']
    pr_nums = []
    priority = 'normal'
    resolution = 'duplicate'
    stage = 'needs patch'
    status = 'closed'
    superseder = '9276'
    type = 'enhancement'
    url = 'https://bugs.python.org/issue558238'
    versions = ['Python 3.2']

    @hinsen
    Copy link
    Mannequin Author

    hinsen mannequin commented May 20, 2002

    Last week I noticed that the pickle and cPickle modules
    cannot handle bound methods. I found a solution that is
    simple and (I think) general, so perhaps it should
    become part of the standard library.

    Here is my code:

    import copy_reg
    
    def pickle_bound_method(method):
        return getattr, (method.im_self, method.__name__)
    
    class _Foo:
        def bar(self):
            pass
    
    _foo = _Foo()
    
    copy_reg.constructor(getattr)
    copy_reg.pickle(type(_foo.bar), pickle_bound_method)

    @hinsen hinsen mannequin assigned loewis May 20, 2002
    @hinsen hinsen mannequin added stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels May 20, 2002
    @hinsen hinsen mannequin assigned loewis May 20, 2002
    @hinsen hinsen mannequin added stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels May 20, 2002
    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Jun 9, 2002

    Logged In: YES
    user_id=21627

    Making getattr a safe_constructor has security implictions
    which make this approach dangerous. It seems that unpickling
    might invoke arbitrary __getattr__ implementations. Adding a
    protocol to declare classes as "safe for getattr" might help.

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Sep 22, 2002

    Logged In: YES
    user_id=21627

    Can you perhaps rewrite this to use new.instancemethod?

    @hinsen
    Copy link
    Mannequin Author

    hinsen mannequin commented Sep 24, 2002

    Logged In: YES
    user_id=11850

    Of course:

    import copy_reg
    
    def get_method(object, name):
        klass = object.__class__
        fn = klass.__dict__[name]
        return new.instancemethod(fn, object, klass)
    
    def pickle_bound_method(method):
        return get_method, (method.im_self, method.__name__)
    
    class _Foo:
        def bar(self):
            pass
    
    _foo = _Foo()
    
    copy_reg.constructor(get_method)
    copy_reg.pickle(type(_foo.bar), pickle_bound_method)

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Sep 24, 2002

    Logged In: YES
    user_id=21627

    That sounds good to me; I'm going to install it.

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Sep 30, 2002

    Logged In: YES
    user_id=21627

    Reconsidering: It fails to work in the presence of
    inheritance, right?

    >>> class B:
    ...   def foo(self):pass
    ... 
    >>> class D(B):pass
    ... 
    >>> d = D()
    >>> m = d.foo
    >>> m.im_self
    <__main__.D instance at 0x1ccb28>
    >>> m.__name__
    'foo'
    >>> D.__dict__['foo']
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    KeyError: foo
    >>>

    @hinsen
    Copy link
    Mannequin Author

    hinsen mannequin commented Sep 30, 2002

    Logged In: YES
    user_id=11850

    True. Would it be a security problem to use getattr inside a
    specialized constructor? For example,

    def get_method(object, name):
        klass = object.__class__
        fn = getattr(klass, name)
        return new.instancemethod(fn, object, klass)

    This would be difficult to abuse as a general getattr
    replacement, because new.instancemethod would fail in most
    cases other than the intended one.

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Sep 30, 2002

    Logged In: YES
    user_id=21627

    My concern is that the getattr call already creates a
    danger, since you just have to find a 'suitable' object
    whose __getattr__ does funny things. In the revised form,
    this object would also have to appear as the __class__ of
    some other object, so I guess this is pretty safe. I'll ask
    python-dev.

    @rhettinger
    Copy link
    Contributor

    Logged In: YES
    user_id=80475

    Any progress on this one?

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Jan 1, 2004

    Logged In: YES
    user_id=21627

    There was some discussion on python-dev (initiated by
    Christian) on whether bound methods should provide an
    __reduce__. Apart from that, no progress.

    @avassalotti
    Copy link
    Member

    Personally, I don't see how adding this feature would create a security
    hole (or more properly said, grow the giant hole that are the GLOBAL and
    REDUCE opcodes). I don't see either why this should be included in the
    standard library.

    Unless there is still a need for this feature, I think this bug should
    be closed.

    @avassalotti
    Copy link
    Member

    I am leaving this issue open for now.

    I reconsidered whether we should add pickle support for methods and I
    now think it would probably be a good idea. For example, the
    multiprocessing module would benefit from a having built-in support for
    method pickling (right now, it has to subclass Pickler to pickle methods).

    @pitrou
    Copy link
    Member

    pitrou commented Jul 16, 2010

    See also bpo-3657. For some reasons, C bound methods aliased as module globals can be pickled, but not similarly aliased Python bound methods:

    >>> random.random, random.seed
    (<built-in method random of Random object at 0x27abe20>, <bound method Random.seed of <random.Random object at 0x27abe20>>)
    >>> pickle.dumps(random.random)
    b'\x80\x03crandom\nrandom\nq\x00.'
    >>> pickle.dumps(random.seed)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/antoine/py3k/__svn__/Lib/pickle.py", line 1314, in dumps
        Pickler(f, protocol, fix_imports=fix_imports).dump(obj)
    _pickle.PicklingError: Can't pickle <class 'method'>: attribute lookup builtins.method failed

    @pitrou
    Copy link
    Member

    pitrou commented Aug 2, 2010

    Let's continue the discussion on bpo-9276.

    @pitrou pitrou closed this as completed Aug 2, 2010
    @pitrou pitrou closed this as completed Aug 2, 2010
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 9, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants