classification
Title: pickle too picky on re-defined classes
Type: crash Stage:
Components: Library (Lib) Versions: Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 2.7, Python 2.6
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, kxroberto
Priority: normal Keywords:

Created on 2011-11-25 09:02 by kxroberto, last changed 2011-12-11 20:18 by gvanrossum. This issue is now closed.

Messages (4)
msg148311 - (view) Author: kxroberto (kxroberto) Date: 2011-11-25 09:02
When a class definition was re-executed (reload, exec ..) , pickling of existing instances fails for to picky reason (class object id mismatch). Solved by the one liner patch below.

Rational: Python is dynamic. Like with any normal attribute lookup: When its the same module/..name  this class is really meant - no matter about its id. 
(During unpickling its another id anyway, the code may have evolved years ...)


diff -ur --strip _orig/pickle.py ./pickle.py
--- _orig/pickle.py	2008-09-08 10:58:32 +0000
+++ ./pickle.py	2011-11-24 15:47:11 +0000
@@ -747,7 +747,7 @@
                 "Can't pickle %r: it's not found as %s.%s" %
                 (obj, module, name))
         else:
-            if klass is not obj:
+            if klass.__name__ != obj.__name__:
                 raise PicklingError(
                     "Can't pickle %r: it's not the same object as %s.%s" %
                     (obj, module, name))
msg148337 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2011-11-25 16:58
Hm, this change allows many other *undesirable* objects pass the test as well. I'd prefer to stick to the rule, "when in doubt, raise an error".
Maybe using == instead of 'is' as the test would be acceptable?
msg149223 - (view) Author: kxroberto (kxroberto) Date: 2011-12-11 11:03
Well, "==" whould allow the wanted feature by exception through meta classes for concerned classes:

>>> class X:
... 	a=1
... 
>>> Y=X
>>> class X:
... 	a=1
... 
>>> Y==X
False
>>> class XCompare(type):
... 	def __eq__(self, other):
... 		print "tolerant class __eq__"
... 		return self.__name__ == other.__name__
... 
>>> class X:
... 	__metaclass__ = XCompare
... 	a=1
... 
>>> Y=X
>>> class X:
... 	a=1
... 
>>> Y==X
tolerant class __eq__
True
>>> 


Better than nothing. Its a improvement generally, independently.
But thinking about my acutal use cases and all: It still doesn't satisfy. I don't want to introduce this extra magic on all those classes just for that feature - because when needed, the majority of classes are concerned (see below). One can have only one meta class ... its too tricky and off-road to guess for most programmers ... 

"when in doubt, raise an error": That is IMHO too rigid here, and generally when a feature is then hindered too much. Aren't warnings the right tool for such case? If really rarely there is problem, should it surface easily already during dev & test time?
Compared to the everday life danger of Pythons dynamic attribute access, version incompatibilities, etc. its about a rather harmless issue here.

Now I'd vote for a warnings.warn upon "==" (or old "is") failing , and then an error only when the .__name__ is not matching too. A warning at dev & test time should be enough, when just "==" (or "is") fails. 


I mainly like the tolerance during development: e.g. fast reload style edit-run cycles (reload sometimes improved with special reload fix code), because I noticed that 95% of code changes/bug fixes do not require a full expensive app-restart. This pays off particularly with bigger GUI app development/fixing and similar, where lot of status is accumulated expensively during run time.
But I wished that feature already for a deployed app too.
msg149250 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2011-12-11 20:18
What you're seeing here is just one of may things that go subtly wrong when you reload a class.  I don't think we should fix this one aspect while leaving so many other bugs due to the same root cause.

It would be better to focus your energy on a way to improve reloading, e.g. make it so that the identity of global functions and classes doesn't change when their module is reloaded.  (You'll find it a tough problem, but note that it's been solved for at least one specific instance: modules *do* retain their identity, so maybe you can use that as a model.)
History
Date User Action Args
2011-12-11 20:18:30gvanrossumsetstatus: open -> closed
resolution: wont fix
messages: + msg149250
2011-12-11 11:03:04kxrobertosetmessages: + msg149223
2011-11-25 16:58:26gvanrossumsetnosy: + gvanrossum
messages: + msg148337
2011-11-25 09:03:34kxrobertosettitle: pickle to picky on re-defined classes -> pickle too picky on re-defined classes
2011-11-25 09:02:42kxrobertocreate