Message356388
If the __qualname__ of a class is set to have a circular reference to itself, pickle behaves differently based on protocol. Following script demonstrates the issue:
======================================================
from __future__ import print_function
import pickle, sys
class Foo:
__name__ = __qualname__ = "Foo.ref"
pass
Foo.ref = Foo
print(sys.version_info)
for proto in range(0, pickle.HIGHEST_PROTOCOL + 1):
print("{}:".format(proto), end=" ")
try:
pkl = pickle.dumps(Foo, proto)
print("Dump OK,", end=" ")
assert(pickle.loads(pkl) is Foo)
print("Load OK,")
except Exception as err:
print(repr(err))
======================================================
OUTPUT:
Python2.7:
sys.version_info(major=2, minor=7, micro=16, releaselevel='final', serial=0)
0: Dump OK, Load OK,
1: Dump OK, Load OK,
2: Dump OK, Load OK,
Python3.7:
sys.version_info(major=3, minor=7, micro=3, releaselevel='final', serial=0)
0: RecursionError('maximum recursion depth exceeded while pickling an object')
1: RecursionError('maximum recursion depth exceeded while pickling an object')
2: RecursionError('maximum recursion depth exceeded while pickling an object')
3: RecursionError('maximum recursion depth exceeded while pickling an object')
4: Dump OK, Load OK,
======================================================
This was introduced as a side effect of issue#23611 (?). I can think of the following approaches to fix the issue and make the behavior consistent:
1. Check if the class has a self-reference and raise an error for all protocols.
2. Use memoization to handle self-references. I am not sure what should be dumped in this case. In the example above `Foo` will exist in the namespace but not `Foo.ref`.
3. Dump such classes similar to Python 2 pickle and Python 3 pickle protocol >= 4.
I had a stab at pickle.py and had a bit of success in doing point 3 above. Posting this issue for discussions. I would be happy to submit a PR for this issue.
Thanks,
Saim Raza |
|
Date |
User |
Action |
Args |
2019-11-11 21:10:18 | Saim Raza | set | recipients:
+ Saim Raza, serhiy.storchaka |
2019-11-11 21:10:17 | Saim Raza | set | messageid: <1573506617.89.0.382822936013.issue38770@roundup.psfhosted.org> |
2019-11-11 21:10:17 | Saim Raza | link | issue38770 messages |
2019-11-11 21:10:17 | Saim Raza | create | |
|