This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author phd
Recipients phd
Date 2013-05-26.18:59:09
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1369594749.9.0.0448045246945.issue18068@psf.upfronthosting.co.za>
In-reply-to
Content
Hello! I've found a problematic behaviour of pickle when it pickles
(direct or indirect) weak proxies to self. I suspect pickle cannot
detect reference loops with weak proxies. I managed to narrow the case
to the following example:

import pickle, weakref

class TestPickle(object):
    def __init__(self):
        self.recursive = weakref.proxy(self)

    def __getstate__(self):
        print "__getstate__", id(self)
        return self.__dict__.copy()

    def __setstate__(self, d):
        print "__setstate__", id(self)
        self.__dict__.update(d)

print "- 1 -"
test = TestPickle()
print "- 2 -"
data = pickle.dumps(test, pickle.HIGHEST_PROTOCOL)
print "- 3 -"
test2 = pickle.loads(data)
print "- 4 -"
print "Result:", id(test2)
print "- 5 -"

   It prints:

- 1 -
- 2 -
__getstate__ 3075348620
__getstate__ 3075348620
- 3 -
__setstate__ 3075348844
__setstate__ 3075349004
- 4 -
Result: 3075349004
- 5 -

   That is, __getstate__ is called twice for the same object. And what
is worse, __setstate__ is called twice for different objects. The
resulting unpickled object is the last one, but in the library that I
have been debugging creation of two different objects during unpickling
is a bug.

   I can fix it by avoiding pickling the proxy and recreating the proxy
on unpickling:

import pickle, weakref

class TestPickle(object):
    def __init__(self):
        self.recursive = weakref.proxy(self)

    def __getstate__(self):
        print "__getstate__", id(self)
        d = self.__dict__.copy()
        del d['recursive']
        return d

    def __setstate__(self, d):
        print "__setstate__", id(self)
        self.__dict__.update(d)
        self.recursive = weakref.proxy(self)

print "- 1 -"
test = TestPickle()
print "- 2 -"
data = pickle.dumps(test, pickle.HIGHEST_PROTOCOL)
print "- 3 -"
test2 = pickle.loads(data)
print "- 4 -"
print "Result:", id(test2)
print "- 5 -"

- 1 -
- 2 -
__getstate__ 3075070092
- 3 -
__setstate__ 3075070188
- 4 -
Result: 3075070188
- 5 -

   But I wonder if it's a bug that should be fixed? If it's an expected
behaviour it perhaps should be documented as a warning in docs for
pickle or weakref or both.
History
Date User Action Args
2013-05-26 18:59:09phdsetrecipients: + phd
2013-05-26 18:59:09phdsetmessageid: <1369594749.9.0.0448045246945.issue18068@psf.upfronthosting.co.za>
2013-05-26 18:59:09phdlinkissue18068 messages
2013-05-26 18:59:09phdcreate