New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
private dispatch table for picklers #58374
Comments
Currently the only documented way to have customised pickling for a type is to register a reduction function with the global dispatch table managed by the copyreg module. But such global changes are liable to disrupt other code which uses pickling. Multiprocessing deals with this by defining a ForkingPickler class which subclasses the pure python _Pickler class (using undocumented features), and supports registering reduction functions specifically for that class. I would like to see some documented alternative which works with both C and Python implementations. At least then multiprocessing can avoid using slow pure python pickling. The attached patch allows a pickler object to have a private dispatch table which it uses *instead* of the global one. It lets one write code like p = pickle.Pickler(...)
p.dispatch_table = copyreg.dispatch_table.copy()
p.dispatch_table[SomeClass] = reduce_SomeClass or class MyPickler(pickle.Pickler):
dispatch_table = copyreg.dispatch_table.copy()
The equivalent using copyreg would be copyreg.pickle(SomeClass, reduce_SomeClass)
p = pickle.Pickler(...) |
That looks like a good idea. + try: ... since self.dispatch_table is a property returning self._dispatch_table. Did you mean type(self).dispatch_table? Also, you need to update the docs (which will also make the intended semantics of the patch clearer :-)). |
More or less. That code was a botched attempt to match the behaviour of the C implementation. The C implementation does not expose the dispatch table unless it has been explicitly set (on the pickler or the pickler class), and it ignores any "dispatch_table" (or "persistent_id") attribute on the metaclass. I will do a fixed patch with docs. |
Updated patch with docs. |
New changeset f7a9a10ae0c0 by Antoine Pitrou in branch 'default': |
I've replaced occurrences of "pickle.Pickler" with "self.pickler_class" in the tests. Otherwise, the patch looks perfect and I've now committed it. Thank you! |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: