classification
Title: Please replace the use of pickle in multiprocessing with json.
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: alex, asdfasdfasdfasdfasdfasdfasdf, asksol, jnoller, r.david.murray, vstinner
Priority: normal Keywords:

Created on 2011-03-01 06:51 by asdfasdfasdfasdfasdfasdfasdf, last changed 2011-03-01 15:03 by asdfasdfasdfasdfasdfasdfasdf. This issue is now closed.

Messages (7)
msg129742 - (view) Author: david (asdfasdfasdfasdfasdfasdfasdf) Date: 2011-03-01 06:51
Please replace the use of pickle in multiprocessing with json.
msg129743 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2011-03-01 07:01
Why?  JSON is incapable of representing most Python datastructures that can be pickled (i.e. anything that isn't a list, tuple, dict, int, or str).
msg129762 - (view) Author: david (asdfasdfasdfasdfasdfasdfasdf) Date: 2011-03-01 13:33
On 1 March 2011 18:01, Alex <report@bugs.python.org> wrote:
>
> Alex <alex.gaynor@gmail.com> added the comment:
>
> Why?  JSON is incapable of representing most Python datastructures that can be pickled (i.e. anything that isn't a list, tuple, dict, int, or str).

I would have suggested yaml (using safe_load() ) . However there isn't
a 'core' yaml module afaik.

If you wanted to actually transfer a complex python data-structure no
one would stop you from pickling it and sending it (json'ed).
msg129779 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-03-01 14:42
This would increase the overhead to no purpose, as far as I can see.

Ask or Jesse can reopen this if they think it is worth considering.
msg129780 - (view) Author: david (asdfasdfasdfasdfasdfasdfasdf) Date: 2011-03-01 14:59
Um ok.
msg129781 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2011-03-01 15:02
I concur with RDM. We need complex data structures, and switching to JSON represents a non zero amount of work, isn't as fast and pickle works well. If you want to use JSON as a transport, I would do custom subclassing.
msg129782 - (view) Author: david (asdfasdfasdfasdfasdfasdfasdf) Date: 2011-03-01 15:02
Fair enough.
History
Date User Action Args
2011-03-01 15:03:00asdfasdfasdfasdfasdfasdfasdfsetnosy: vstinner, jnoller, alex, r.david.murray, asksol, asdfasdfasdfasdfasdfasdfasdf
messages: + msg129782
2011-03-01 15:02:11jnollersetnosy: vstinner, jnoller, alex, r.david.murray, asksol, asdfasdfasdfasdfasdfasdfasdf
messages: + msg129781
2011-03-01 14:59:40asdfasdfasdfasdfasdfasdfasdfsetnosy: vstinner, jnoller, alex, r.david.murray, asksol, asdfasdfasdfasdfasdfasdfasdf
messages: + msg129780
2011-03-01 14:42:57r.david.murraysetstatus: open -> closed

nosy: + asksol, r.david.murray, jnoller
messages: + msg129779

resolution: rejected
2011-03-01 14:01:03vstinnersetnosy: + vstinner
2011-03-01 13:33:18asdfasdfasdfasdfasdfasdfasdfsetmessages: + msg129762
2011-03-01 07:54:50ezio.melottisetversions: + Python 3.3
type: enhancement
components: + Library (Lib)
stage: needs patch
2011-03-01 07:53:32ezio.melottilinkissue11359 superseder
2011-03-01 07:01:42alexsetnosy: + alex
messages: + msg129743
2011-03-01 06:51:41asdfasdfasdfasdfasdfasdfasdfcreate