classification
Title: python pickle.dumps AssertionError
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.6
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Cannot pickle self-referencing sets
View: 9269
Assigned To: Nosy List: Leo.Na, belopolsky
Priority: normal Keywords:

Created on 2010-12-14 05:04 by Leo.Na, last changed 2010-12-14 14:55 by belopolsky. This issue is now closed.

Files
File name Uploaded Description Edit
set-graph.py belopolsky, 2010-12-14 14:45
dict-graph.py belopolsky, 2010-12-14 14:45
Messages (4)
msg123925 - (view) Author: HwidongNa (Leo.Na) Date: 2010-12-14 05:04
I'm trying to pickle a class instance containing two lists of another instances. The instances in the two lists have attributes that refer instances of each other. Here are the classes.

import pickle
from copy import copy

class Graph:
    def __init__(self):
        self.vertices = {}
        self.edges = set()
    def __repr__(self):
        return "\n".join(map(str, sorted(self.vertices, key=lambda v:v.id)))

class Edge:
    def __init__(self, vfrom, vto):
        self.vfrom = vfrom
        self.vto = vto
    def __hash__(self):
        return hash((self.vto, self.vfrom))
    def __repr__(self):
        return str(self.vto.id)
    def __getstate__(self):
        vfrom = copy(self.vfrom)
        vfrom.del_outgoing(self)
        vto = copy(self.vto)
        vto.del_incoming(self)
        self.__dict__.update({"vfrom":vfrom, "vto":vto, })
        return self.__dict__
    def __setstate__(self, state):
        self.__dict__.update(state)
        self.__dict__["vfrom"].add_outgoing(self)
        self.__dict__["vto"].add_incoming(self)

class Vertex:
    def __init__(self, id):
        self.id = id
        self.incoming = set()
        self.outgoing = set()
    def __repr__(self):
        return "Vertex %d -> %s"%(self.id, ", ".join(map(str, self.outgoing)))
    def __hash__(self):
        return hash(self.id)
    def add_incoming(self, edge):
        if not edge in self.incoming:
            self.incoming.add(edge)
    def add_outgoing(self, edge):
        if not edge in self.outgoing:
            self.outgoing.add(edge)
    def del_incoming(self, edge):
        self.incoming.discard(edge)
    def del_outgoing(self, edge):
        self.outgoing.discard(edge)

I got an AssertionError when I pickled a simple graph as follows.

>>> v0 = Vertex(0)
>>> v1 = Vertex(1)
>>> e0to1 = Edge(v0, v1)
>>> v0.add_outgoing(e0to1)
>>> v1.add_incoming(e0to1)
>>> g = Graph()
>>> g.vertices[v0] = v0
>>> g.vertices[v1] = v1
>>> g.edges.add(e0to1)
>>> g.edges.add(e0to1)
>>> v2 = Vertex(2)
>>> e0to2 = Edge(v0, v2)
>>> v0.add_outgoing(e0to2)
>>> v2.add_incoming(e0to2)
>>> g.vertices[v2] = v2
>>> g.edges.add(e0to2)
>>> 
>>> print g
Vertex 0 -> 2, 1
Vertex 1 -> 
Vertex 2 -> 
>>> p = pickle.dumps(g)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/pickle.py", line 1366, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib/python2.6/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.6/pickle.py", line 725, in save_inst
    save(stuff)
  File "/usr/lib/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.6/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.6/pickle.py", line 663, in _batch_setitems
    save(v)
  File "/usr/lib/python2.6/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.6/pickle.py", line 401, in save_reduce
    save(args)
  File "/usr/lib/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.6/pickle.py", line 562, in save_tuple
    save(element)
  File "/usr/lib/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.6/pickle.py", line 600, in save_list
    self._batch_appends(iter(obj))
  File "/usr/lib/python2.6/pickle.py", line 615, in _batch_appends
    save(x)
  File "/usr/lib/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.6/pickle.py", line 725, in save_inst
    save(stuff)
  File "/usr/lib/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.6/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.6/pickle.py", line 663, in _batch_setitems
    save(v)
  File "/usr/lib/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.6/pickle.py", line 725, in save_inst
    save(stuff)
  File "/usr/lib/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.6/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.6/pickle.py", line 663, in _batch_setitems
    save(v)
  File "/usr/lib/python2.6/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.6/pickle.py", line 405, in save_reduce
    self.memoize(obj)
  File "/usr/lib/python2.6/pickle.py", line 244, in memoize
    assert id(obj) not in self.memo
AssertionError

I worked when the v2 is removed. 

>>> v0 = Vertex(0)
>>> v1 = Vertex(1)
>>> e0to1 = Edge(v0, v1)
>>> v0.outgoing.add(e0to1)
>>> v1.incoming.add(e0to1)
>>> g = Graph()
>>> g.vertices[v0] = v0
>>> g.vertices[v1] = v1
>>> g.edges.add(e0to1)
>>> g.edges.add(e0to1)
>>> import cPickle as pickle
>>> p = pickle.dumps(g)
>>> print pickle.loads(p)
Vertex 0 -> 1
Vertex 1 -> 

Do you have any idea?
msg123926 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-12-14 05:32
This looks like another duplicate of issue1062277.
msg123947 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-12-14 14:45
This is a known problem. See issues #1062277 and #9269.  You can work around the issue by using a dict.  I am attaching two test files. First, set-graph.py, reproduces the issue in 3.x and the second, dict-graph.py, contains a workaround.
msg123950 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-12-14 14:55
It turns out that dict-graph.py does not work with python2.x, but that is a different problem, so I opened a separate issue for it.  See issue10701.
History
Date User Action Args
2010-12-14 14:55:27belopolskysetmessages: + msg123950
2010-12-14 14:45:52belopolskysetfiles: + dict-graph.py
2010-12-14 14:45:33belopolskysetstatus: open -> closed
files: + set-graph.py
superseder: Cannot pickle self-referencing sets

type: crash -> behavior
messages: + msg123947
resolution: duplicate
stage: resolved
2010-12-14 05:32:11belopolskysetnosy: + belopolsky
messages: + msg123926
2010-12-14 05:04:53Leo.Nacreate