This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Saim Raza
Recipients Saim Raza, serhiy.storchaka
Date 2019-11-11.21:10:17
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1573506617.89.0.382822936013.issue38770@roundup.psfhosted.org>
In-reply-to
Content
If the __qualname__ of a class is set to have a circular reference to itself, pickle behaves differently based on protocol. Following script demonstrates the issue:

======================================================
from __future__ import print_function

import pickle, sys

class Foo:
    __name__ = __qualname__ = "Foo.ref"
    pass

Foo.ref = Foo

print(sys.version_info)

for proto in range(0, pickle.HIGHEST_PROTOCOL + 1):
    print("{}:".format(proto), end=" ")
    try:
        pkl = pickle.dumps(Foo, proto)
        print("Dump OK,", end=" ")
        assert(pickle.loads(pkl) is Foo)
        print("Load OK,")
    except Exception as err:
        print(repr(err))
======================================================
OUTPUT:
Python2.7:
sys.version_info(major=2, minor=7, micro=16, releaselevel='final', serial=0)
0: Dump OK, Load OK,
1: Dump OK, Load OK,
2: Dump OK, Load OK,

Python3.7:
sys.version_info(major=3, minor=7, micro=3, releaselevel='final', serial=0)
0: RecursionError('maximum recursion depth exceeded while pickling an object')
1: RecursionError('maximum recursion depth exceeded while pickling an object')
2: RecursionError('maximum recursion depth exceeded while pickling an object')
3: RecursionError('maximum recursion depth exceeded while pickling an object')
4: Dump OK, Load OK,
======================================================

This was introduced as a side effect of issue#23611 (?). I can think of the following approaches to fix the issue and make the behavior consistent:

1. Check if the class has a self-reference and raise an error for all protocols.
2. Use memoization to handle self-references. I am not sure what should be dumped in this case. In the example above `Foo` will exist in the namespace but not `Foo.ref`. 
3. Dump such classes similar to Python 2 pickle and Python 3 pickle protocol >= 4.

I had a stab at pickle.py and had a bit of success in doing point 3 above. Posting this issue for discussions. I would be happy to submit a PR for this issue.

Thanks,
Saim Raza
History
Date User Action Args
2019-11-11 21:10:18Saim Razasetrecipients: + Saim Raza, serhiy.storchaka
2019-11-11 21:10:17Saim Razasetmessageid: <1573506617.89.0.382822936013.issue38770@roundup.psfhosted.org>
2019-11-11 21:10:17Saim Razalinkissue38770 messages
2019-11-11 21:10:17Saim Razacreate