Author herring
Recipients ajaksu2, alex, alexandre.vassalotti, bcroq, belopolsky, ddorfman, eltoder, herring, mstefanro, pitrou, rhettinger, zzzeek
Date 2015-11-21.08:17:58
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
Re msg110868: it's impossible to resolve a __reduce__ loop that involves only immutable intermediate objects (including none at all):

class Direct:
  def __reduce__(self): return id,(self,) # obviously impossible
class Indirect:
  # Can't create either the new object or the tuple first!
  def __reduce__(self): return id,((self,),)

With pre-existing mutable objects, the same trick as for tuples certainly could be applied:

class Mutable:
  def __init__(self): self.bracketed=[self]
  # Create list, REDUCE, then populate list
  def __reduce__(self): return id,(self.bracketed,)

The way an analog to the tuple implementation would deal with this would be something like

id   # the dummy "constructor" in this example
[]   # empty list, memoized immediately as, say, #5
id   # try again to store the Mutable
@5   # fetch from memo
T1   # make singleton tuple
R    # reduce (have now succeeded in "making" the Mutable as, say, #20)
APP  # to the list
T1   # finish the first __reduce__ attempt
POP  # abandon it, since the object has been created
POP  # abandon the "id" as well
@20  # fetch the previous answer

If the list were created directly in __reduce__, you'd still recurse infinitely trying to create each new list object.  On the other hand, even complicated cases like this should work:

class Switch:
  def __reduce__(self): return id,(self.lst,)
a=Switch(); b=Switch()
a.list=[b,a]; b.list=[a,b]

where the pickling of, say, a would look something like

[]    # -> #17
id    # trying b now
[]    # -> #42
id    # trying a again
R     # a done (#88)
id    # trying b again
R     # b done (#101)
APP   # b's list now populated
POP   # b abandoned
@101  # b fetched
APP   # finally building a's list
@88   # a is long done
APP   # a's list now populated
POP   # a abandoned
@88   # final value: a

Perhaps a technique for distinguishing these cases is to look for new objects having been created since the last time we visited an object.  If there are none, we're in the immutable case and we lose.  If there are yet more created between the second and third visits, on the other hand, we're generating more objects from __reduce__ every time and should again abort.
Date User Action Args
2015-11-21 08:17:59herringsetrecipients: + herring, rhettinger, belopolsky, ddorfman, pitrou, ajaksu2, alexandre.vassalotti, alex, zzzeek, bcroq, eltoder, mstefanro
2015-11-21 08:17:59herringsetmessageid: <>
2015-11-21 08:17:59herringlinkissue1062277 messages
2015-11-21 08:17:58herringcreate