This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Support pickling slots in subclasses of common classes
Type: enhancement Stage: resolved
Components: Extension Modules, Library (Lib) Versions: Python 3.11
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: alexandre.vassalotti, berker.peksag, pitrou, r.david.murray, rhettinger, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2016-03-17 08:43 by serhiy.storchaka, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
copyreg_getstate.patch serhiy.storchaka, 2016-03-17 08:43 review
copyreg_getstate2.patch serhiy.storchaka, 2016-06-18 19:59 review
object_getstate.patch serhiy.storchaka, 2016-07-14 11:50 Add object.__getstate__ review
Pull Requests
URL Status Linked Edit
PR 2821 merged serhiy.storchaka, 2017-07-23 08:32
Messages (5)
msg261903 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-03-17 08:43
Pickling and copying instances of subclasses of some basic classes pickles and copies instance attributes. Example:

>>> class BA(bytearray):
...     pass
... 
>>> b = BA(b'abc')
>>> b.x = 10
>>> c = copy.copy(b)
>>> c.x
10
>>> c = pickle.loads(pickle.dumps(b))
>>> c.x
10

But this doesn't work if attributes are saved not in instance dictionary, but in slots.

>>> class BA(bytearray):
...     __slots__ = ('x',)
... 
>>> b = BA(b'abc')
>>> b.x = 10
>>> c = copy.copy(b)
>>> c.x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: x
>>> c = pickle.loads(pickle.dumps(b))
>>> c.x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: x

Since using __slots__ is implementation detail, this failure can be considered as a bug.

Proposed patch adds support of pickling and copying slots in subclasses of all classes that already support pickling and copying non-slot attributes. It is backward compatible, classes with slots can be unpickled on older Python versions without slots. Affected classes: bytearray, set, frozenset, weakref.WeakSet, collections.OrderedDict, collections.deque, datetime.tzinfo.

The patch adds the copyreg._getstate() function for Python classes and exposes the _PyObject_GetState() function for extension classes. An alternative (and simpler for end user) solution would be to add default implementation as object.__getstate__(). But this is not easy to reject non-pickleable classes (issue22995) in this case, since __getstate__ is looked up as instance attribute, not as other special methods.
msg268830 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-06-18 19:59
Synchronized with current sources.
msg270403 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-07-14 11:50
An alternative way is to expose a default state as object.__getstate__(). It is more efficient since it is implemented in C. Following patch implements this approach.
msg416888 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2022-04-06 17:00
New changeset 884eba3c76916889fd6bff3b37b8552bfb4f9566 by Serhiy Storchaka in branch 'main':
bpo-26579: Add object.__getstate__(). (GH-2821)
https://github.com/python/cpython/commit/884eba3c76916889fd6bff3b37b8552bfb4f9566
msg416931 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-04-07 14:57
> bpo-26579: Add object.__getstate__(). (GH-2821)
> https://github.com/python/cpython/commit/884eba3c76916889fd6bff3b37b8552bfb4f9566

This change introduced reference leaks: see bpo-47250.
History
Date User Action Args
2022-04-11 14:58:28adminsetgithub: 70766
2022-04-07 14:57:28vstinnersetmessages: + msg416931
2022-04-06 17:02:02serhiy.storchakasetstatus: open -> closed
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.11, - Python 3.7
2022-04-06 17:00:19serhiy.storchakasetmessages: + msg416888
2020-08-14 06:25:45serhiy.storchakalinkissue41547 superseder
2017-11-15 21:15:23serhiy.storchakasetpull_requests: - pull_request4360
2017-11-15 21:14:27yselivanovsetpull_requests: + pull_request4360
2017-07-23 08:33:50serhiy.storchakasetnosy: + vstinner, r.david.murray

versions: + Python 3.7, - Python 3.6
2017-07-23 08:32:55serhiy.storchakasetpull_requests: + pull_request2873
2016-07-14 11:50:18serhiy.storchakasetfiles: + object_getstate.patch

messages: + msg270403
2016-06-18 19:59:45serhiy.storchakasetfiles: + copyreg_getstate2.patch

messages: + msg268830
2016-03-19 12:04:32berker.peksagsetnosy: + berker.peksag
2016-03-17 08:43:06serhiy.storchakacreate