New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cPickle to pickle conversion in py3k missing methods #47635
Comments
I was attempting the patch for bpo-3125 to py3k, and in it Amaury from pickle import Pickler
class ForkingPickler(Pickler):
dispatch = Pickler.dispatch.copy() This is also related to bpo-3350 I suspect. However, using the Trunk:
Python 2.6b1+ (trunk:65015M, Jul 16 2008, 10:15:51)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> dir(pickle.Pickler)
['_BATCHSIZE', '__doc__', '__init__', '__module__', '_batch_appends',
'_batch_setitems', 'clear_memo', 'dispatch', 'dump', 'get', 'memoize',
'persistent_id', 'put', 'save', 'save_bool', 'save_dict',
'save_empty_tuple', 'save_float', 'save_global', 'save_inst',
'save_int', 'save_list', 'save_long', 'save_none', 'save_pers',
'save_reduce', 'save_string', 'save_tuple', 'save_unicode']
>>>
py3k:
Python 3.0b1+ (py3k:65011M, Jul 16 2008, 11:50:11)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> dir(Pickler)
['__class__', '__delattr__', '__doc__', '__eq__', '__format__',
'__ge__', '__getattribute__', '__gt__', '__hash__', '__init__',
'__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__',
'__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__',
'bin', 'clear_memo', 'dump', 'fast', 'memo', 'persistent_id'] I think the fix for 3125 resolves your complaint in 3350, but is the |
The omission of the dispatch dictionary was sort of intentional. But |
Just in case you are wondering why I haven't submitted a patch yet, I |
A use case: |
I got a preliminary patch that adds the dispatch dictionary. However, |
I ran into a few problems while trying to fix this issue. First, does Second, which methods of Pickler should be made public? I know Third, should Pickler allows pickling support for built-in types (e.g., |
Maybe I missed something, but the (subclassable) python implementation from pickle import _Pickler as Pickler
class ForkingPickler(Pickler):
dispatch = Pickler.dispatch.copy() |
Yeah, the old Pickler and Unpickler classes are available by design (to |
Do I understand correctly that the issue is that python Pickler class has dispatch attribute but C Pickler does not? The add_dispatch_check-0.patch patch does not seem to add class attribute, it adds an instance attribute instead. I also noticed that there are many more class attributes that only Python implementation has: >>> for x in dir(pickle._Pickler):
... if not (x.startswith('_') or hasattr(pickle.Pickler, x)):
... print(x)
...
dispatch
get
memoize
put
save
save_bool
save_bytes
save_dict
save_float
save_global
save_list
save_long
save_none
save_pers
save_reduce
save_str
save_tuple The save_* methods are clearly internal and should probably be renamed to begin with __ unless they are intended to be overridden by Pickler subclases |
Yes.
I know. That's why I said it was a preliminary patch. :-) Also, see msg71159 about the class vs instance attribute issue. There's a lot of code out there that do crazy things with pickle's dispatch dictionary. For now, it is best if we leave the method names alone. If people wants to keep doing their funky stuff with pickle, then we can point them to pickle._Pickler and pickle._Unpickler, which will always point to the Python implementations. That said, I do not believe it's a good idea to add the dispatch attribute to _pickle anymore. But, I acknowledge there is a need for overriding the pickling mechanism for specific types. We should find a clean way to support this. I am closing this issue as "won't fix". |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: