Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pickle bug - recursively memoizing class? #40652

Closed
smontanaro opened this issue Jul 27, 2004 · 8 comments
Closed

pickle bug - recursively memoizing class? #40652

smontanaro opened this issue Jul 27, 2004 · 8 comments
Assignees
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@smontanaro
Copy link
Contributor

BPO 998998
Nosy @abalkin, @avassalotti
Superseder
  • bpo-9269: Cannot pickle self-referencing sets
  • Files
  • picklecycle.py
  • memoize.py
  • cycle.out
  • picklecycle3.py
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/abalkin'
    closed_at = <Date 2010-08-16.20:02:38.148>
    created_at = <Date 2004-07-27.20:55:41.000>
    labels = ['type-bug', 'library']
    title = 'pickle bug - recursively memoizing class?'
    updated_at = <Date 2010-08-16.20:02:38.147>
    user = 'https://github.com/smontanaro'

    bugs.python.org fields:

    activity = <Date 2010-08-16.20:02:38.147>
    actor = 'belopolsky'
    assignee = 'belopolsky'
    closed = True
    closed_date = <Date 2010-08-16.20:02:38.148>
    closer = 'belopolsky'
    components = ['Library (Lib)']
    creation = <Date 2004-07-27.20:55:41.000>
    creator = 'skip.montanaro'
    dependencies = []
    files = ['1354', '1355', '1356', '18008']
    hgrepos = []
    issue_num = 998998
    keywords = []
    message_count = 8.0
    messages = ['21827', '21828', '21829', '21830', '77200', '88576', '110343', '110400']
    nosy_count = 5.0
    nosy_names = ['belopolsky', 'grubert', 'alexandre.vassalotti', 'schmir', 'zzzeek']
    pr_nums = []
    priority = 'normal'
    resolution = 'duplicate'
    stage = 'needs patch'
    status = 'closed'
    superseder = '9269'
    type = 'behavior'
    url = 'https://bugs.python.org/issue998998'
    versions = ['Python 3.2']

    @smontanaro
    Copy link
    Contributor Author

    The attached script (picklecycle.py) gives me an
    assertion error
    in Pickler.memoize(). Maybe I'm just doing something dumb.
    I have very little experience with pickle and even less
    experience
    pickling new-style class instances. I'm just trying to
    adapt an
    existing messy object graph to allow it to be pickled.
    I realize the
    two instances refer to each other, but I thought pickle was
    supposed to gracefully handle cyclic object graphs.

    I can worm around the problem a couple ways. First,
    getting
    rid of new-style classes does the trick. Also
    modifying B.__reduce__
    to not return self.__dict__ seems to do the trick.

    Output (cycle.out) and a modified version of
    Pickler.memoize
    (memoize.py) will be attached momentarily.

    @smontanaro smontanaro added stdlib Python modules in the Lib dir labels Jul 27, 2004
    @smontanaro
    Copy link
    Contributor Author

    Logged In: YES
    user_id=44345

    Attaching chatty version of Pickler.memoize()

    @smontanaro
    Copy link
    Contributor Author

    Logged In: YES
    user_id=44345

    Attaching the output I see when running picklecycle.py w/ the
    modified memoize() method (cycle.out).

    @grubert
    Copy link
    Mannequin

    grubert mannequin commented May 19, 2005

    Logged In: YES
    user_id=147070

    it does memoize recursively,
    but "asserts" not to get stuck in cycles.

    that's how i understand the code if ::

        assert id(obj) not in self.memo
    

    is replaced by ::

            if id(obj) in self.memo:
                return

    it works (better). and test_pickle.py still passes.

    @zzzeek
    Copy link
    Mannequin

    zzzeek mannequin commented Dec 7, 2008

    This bug can be reproduced very easily:

        import pickle
    
        class MyClass(object):
            pass
        
        m = MyClass()
        m2 = MyClass()
    
        s = set([m])
        m.foo = set([m2])
        m2.foo = s
    print pickle.dumps(s)
    

    This bug is critical as the pure-python pickle module is required when
    trying to remove warnings in -3 mode with Python 2.6, which is something
    that will be very common in the near future. It's easily reproducible
    with simplistic object cycles like that of the above, in 2.5, 2.6 and
    3.0 at least.

    @zzzeek
    Copy link
    Mannequin

    zzzeek mannequin commented May 30, 2009

    im noticing my test case seems to work fine in py 3.0.1. haven't
    tested 2.6.2.

    @abalkin
    Copy link
    Member

    abalkin commented Jul 15, 2010

    This looks like a duplicate of bpo-1581183. The issue is still present in py3k, but only shows up in picklecycle.py when using pickle.py version. I am attaching picklecycle3.py which is py3k version of picklecycle.py using python pickler.

    @abalkin abalkin self-assigned this Jul 15, 2010
    @abalkin abalkin added the type-bug An unexpected behavior, bug, or error label Jul 15, 2010
    @abalkin abalkin self-assigned this Jul 15, 2010
    @abalkin abalkin added the type-bug An unexpected behavior, bug, or error label Jul 15, 2010
    @abalkin
    Copy link
    Member

    abalkin commented Jul 15, 2010

    There are two issues here. The original issue is a duplicate of bpo-1062277. The other issue is specific to cycles containing a set. [msg77200]

    I am opening a separate issue for that. See bpo-9269.

    @abalkin abalkin closed this as completed Aug 16, 2010
    @abalkin abalkin closed this as completed Aug 16, 2010
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 9, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants