classification
Title: Can't pickle partial functions
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: alexandre.vassalotti, danhs, flox, jcea, ngrilly, pythonmeister
Priority: normal Keywords:

Created on 2007-11-07 15:27 by danhs, last changed 2011-12-11 01:28 by jcea. This issue is now closed.

Files
File name Uploaded Description Edit
partial_bug.py danhs, 2007-11-07 15:27
Messages (6)
msg57200 - (view) Author: Daniel (danhs) Date: 2007-11-07 15:27
Creating a function using functools.partial results in a function which
cannot be pickled.

Attached is a small testcase.
msg57260 - (view) Author: Stefan Sonnenberg-Carstens (pythonmeister) Date: 2007-11-08 16:49
You are using an old protocol version

pickle.dumps(partial_f,2)

does the trick:

>>> pickle.dumps(partial_f,2)
'\x80\x02cfunctools\npartial\nq\x00)\x81q\x01}q\x02b.'
msg71171 - (view) Author: Nicolas Grilly (ngrilly) Date: 2008-08-15 15:16
It seems using protocol version 2 is not enough:

>>> s = pickle.dumps(partial_f, 2)
>>> f = pickle.loads(s)
Traceback (most recent call last):
...
TypeError: type 'partial' takes at least one argument

Am I missing something?
msg71177 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2008-08-15 18:33
I agree that this a bug. However, the liberal functools.partial
constructor makes it hard to pickle partial instances correctly.
Ideally, we would add the __getnewargs__ special method and be done with
it. But, this won't work in this case due to the *args and **kwargs
arguments of partial. Since pickle supports neither, we would need to
use apply(), which going to be removed in Python 3.0, as follow:

>>> import pickletools
>>>
pickletools.dis("c__builtin__\napply\n(cfunctools\npartial\n(c__main__\nf\nt(S'b'\nK\x01dtR.")
    0: c    GLOBAL     '__builtin__ apply'
   19: (    MARK
   20: c        GLOBAL     'functools partial'
   39: (        MARK
   40: c            GLOBAL     '__main__ f'
   52: t            TUPLE      (MARK at 39)
   53: (        MARK
   54: S            STRING     'b'
   59: K            BININT1    1
   61: d            DICT       (MARK at 53)
   62: t        TUPLE      (MARK at 19)
   63: R    REDUCE
   64: .    STOP

Unfortunately, pickle.Pickler cannot generate a such pickle stream. So
this bug is symptom of the bigger issue that classes with *args and/or
**kwargs argument cannot be made picklable.
msg95940 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2009-12-03 19:23
It seems fixed on trunk.
It is fixed on Python 3.1 and 3.2, too.


With Python 2.6 I see the error:
>>> ./python partial_bug.py
TypeError: can't pickle partial objects


Documentation for "trunk" and "py3k" is OK?
msg96254 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2009-12-11 13:34
Indeed. Jack Diederich added support for pickling partial functions in
r70931.
History
Date User Action Args
2011-12-11 01:28:15jceasetnosy: + jcea
2009-12-11 13:34:57alexandre.vassalottisetstatus: open -> closed
resolution: fixed
messages: + msg96254

stage: resolved
2009-12-03 19:23:04floxsetnosy: + flox
messages: + msg95940
2008-08-15 18:33:37alexandre.vassalottisetstatus: closed -> open
priority: normal
resolution: not a bug -> (no value)
messages: + msg71177
nosy: + alexandre.vassalotti
2008-08-15 15:16:50ngrillysetnosy: + ngrilly
messages: + msg71171
2007-11-08 17:34:55christian.heimessetstatus: open -> closed
resolution: not a bug
2007-11-08 16:49:26pythonmeistersetnosy: + pythonmeister
messages: + msg57260
2007-11-07 15:27:30danhscreate