This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Cannot pickle a class with a metaclass
Type: Stage:
Components: Interpreter Core Versions: Python 2.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gvanrossum Nosy List: gvanrossum, mathematician, tim.peters
Priority: high Keywords:

Created on 2001-12-19 04:52 by mathematician, last changed 2022-04-10 16:04 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
pickle.patch gvanrossum, 2001-12-19 06:23
Messages (7)
msg8326 - (view) Author: Dan Parisien (mathematician) Date: 2001-12-19 04:52
when pickle retrieves the __reduce__ method of a new 
style class that has a metaclass, instead of 
returning the metaclass's __reduce__ method bound to 
the class, it returns an unbound __reduce__ method of 
that class.

>>> class metaclass(type):
... 	def __reduce__(self):
... 		"""This is metaclass.__reduce__
... 		"""
... 		return type.__reduce__(self)
... 
>>> class newclass(object):
... 	__metaclass__ = metaclass
... 	def __reduce__(self):
... 		"""This is newclass.__reduce__
... 		"""
... 		return object.__reduce__(self)
... 
>>> print newclass.__reduce__.__doc__
This is newclass.__reduce__

when pickle calls object.__reduce__ on newclass, it 
returns an unbound newclass.__reduce__ and not a 
bound metaclass.__reduce__. This has the unfortunate 
side effect of not correctly 'reducing' the class. 
I'm trying to figure out a solution. 

msg8327 - (view) Author: Dan Parisien (mathematician) Date: 2001-12-19 05:00
Logged In: YES 
user_id=118203

it's a shame the tabs do not appear above...
--- Solution

class metaclass(type):
    def __getattribute__(self, name):
        if name in ["__reduce__", "__getstate__", 
"__setstate__"]:
            return lambda s=self, f=getattr(type(self), 
name): f(s)
        return type.__getattribute__(self, name)

this fixed my bug, but it may not work for everybody. My 
suggestion is if you are to pickle a new style class, you 
should call 
type(new_style_class).__reduce__(new_style_class) instead 
of new_style_class.__reduce__()
msg8328 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2001-12-19 05:19
Logged In: YES 
user_id=31435

Assigned to Guido.

Dan, leading tabs vanish from the web view, but are visible 
in auto-emailed versions (it's a display issue, it's not 
that the database lost them) -- so don't worry about that.
msg8329 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2001-12-19 05:46
Logged In: YES 
user_id=6380

Very interesting!  Good analysis too (took me a while with
pdb to come to verify it :-). But neither class needs to
define __reduce__ -- the one they inherit from their bases,
type and object, will do the trick just as well.

The problem is that there are two things competing to be
newclass.__reduce__: on the one hand, the unbound method
__reduce__ of newclass instances; on the other hand, the
bound method __reduce__ of newclass, seen as a metaclass
instance. Unfortunately, the unbound method wins, but pickle
is expecting the bound method.

I've tried experimenting with a few ways of fixing this
(e.g. forcing pickle to get the bound __reduce__ method) but
there seem to be powerful forces preventing me from getting
this to work (and I don't mean just a screaming baby :-). I
think maybe the right solution is to make pickle realize
that newclass is a class, and prevent it from pickling it at
all -- it should just pickle a reference to it (i.e. the
module and class name) rather than attempting to pickle its
contents.

Stay tuned.
msg8330 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2001-12-19 06:23
Logged In: YES 
user_id=6380

I've got a patch for pickle.py; a similar patch needs to be
developed for cPickle.c.  I can't find a way to fix this
purely by fixing the type object implementation -- pickle.py
and cPickle.py just don't recognize classes with a custom
metaclass as classes, because of their roots in pre-2.2
patterns. :-(

Should we risk fixing this before 2.2 goes out?
msg8331 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2001-12-19 07:00
Logged In: YES 
user_id=31435

Sounds like I'd risk it, Guido:  it's not like you could 
break 2.1's pickle behavior for new-style classes with a 
custom metaclass <wink>.
msg8332 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2001-12-19 16:58
Logged In: YES 
user_id=6380

Thanks for reporting! Fixed in CVS, with a simpler version
of the patch below; also for cPickle. Adding tests too...
History
Date User Action Args
2022-04-10 16:04:48adminsetgithub: 35791
2001-12-19 04:52:18mathematiciancreate