Author serhiy.storchaka
Recipients alexandre.vassalotti, pitrou, serhiy.storchaka
Date 2013-04-13.11:41:49
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1365853309.55.0.468755952624.issue17711@psf.upfronthosting.co.za>
In-reply-to
Content
Python 2 allows pickling and unpickling non-ascii persistent ids. In Python 3 C implementation of pickle saves persistent ids with protocol version 0 as utf8-encoded strings and loads as bytes.

>>> import pickle, io
>>> class MyPickler(pickle.Pickler):
...     def persistent_id(self, obj):
...         if isinstance(obj, str):
...             return obj
...         return None
... 
>>> class MyUnpickler(pickle.Unpickler):
...     def persistent_load(self, pid):
...         return pid
... 
>>> f = io.BytesIO(); MyPickler(f).dump('\u20ac'); data = f.getvalue()
>>> MyUnpickler(io.BytesIO(data)).load()
'€'
>>> f = io.BytesIO(); MyPickler(f, 0).dump('\u20ac'); data = f.getvalue()
>>> MyUnpickler(io.BytesIO(data)).load()
b'\xe2\x82\xac'
>>> f = io.BytesIO(); MyPickler(f, 0).dump('a'); data = f.getvalue()
>>> MyUnpickler(io.BytesIO(data)).load()
b'a'

Python implementation in Python 3 doesn't works with non-ascii persistant ids at all.
History
Date User Action Args
2013-04-13 11:41:49serhiy.storchakasetrecipients: + serhiy.storchaka, pitrou, alexandre.vassalotti
2013-04-13 11:41:49serhiy.storchakasetmessageid: <1365853309.55.0.468755952624.issue17711@psf.upfronthosting.co.za>
2013-04-13 11:41:49serhiy.storchakalinkissue17711 messages
2013-04-13 11:41:49serhiy.storchakacreate