classification
Title: Support pickling slots in subclasses of common classes
Type: enhancement Stage: patch review
Components: Extension Modules, Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: alexandre.vassalotti, berker.peksag, pitrou, r.david.murray, rhettinger, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2016-03-17 08:43 by serhiy.storchaka, last changed 2017-11-15 21:15 by serhiy.storchaka.

Files
File name Uploaded Description Edit
copyreg_getstate.patch serhiy.storchaka, 2016-03-17 08:43 review
copyreg_getstate2.patch serhiy.storchaka, 2016-06-18 19:59 review
object_getstate.patch serhiy.storchaka, 2016-07-14 11:50 Add object.__getstate__ review
Pull Requests
URL Status Linked Edit
PR 2821 open serhiy.storchaka, 2017-07-23 08:32
Messages (3)
msg261903 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-03-17 08:43
Pickling and copying instances of subclasses of some basic classes pickles and copies instance attributes. Example:

>>> class BA(bytearray):
...     pass
... 
>>> b = BA(b'abc')
>>> b.x = 10
>>> c = copy.copy(b)
>>> c.x
10
>>> c = pickle.loads(pickle.dumps(b))
>>> c.x
10

But this doesn't work if attributes are saved not in instance dictionary, but in slots.

>>> class BA(bytearray):
...     __slots__ = ('x',)
... 
>>> b = BA(b'abc')
>>> b.x = 10
>>> c = copy.copy(b)
>>> c.x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: x
>>> c = pickle.loads(pickle.dumps(b))
>>> c.x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: x

Since using __slots__ is implementation detail, this failure can be considered as a bug.

Proposed patch adds support of pickling and copying slots in subclasses of all classes that already support pickling and copying non-slot attributes. It is backward compatible, classes with slots can be unpickled on older Python versions without slots. Affected classes: bytearray, set, frozenset, weakref.WeakSet, collections.OrderedDict, collections.deque, datetime.tzinfo.

The patch adds the copyreg._getstate() function for Python classes and exposes the _PyObject_GetState() function for extension classes. An alternative (and simpler for end user) solution would be to add default implementation as object.__getstate__(). But this is not easy to reject non-pickleable classes (issue22995) in this case, since __getstate__ is looked up as instance attribute, not as other special methods.
msg268830 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-06-18 19:59
Synchronized with current sources.
msg270403 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-07-14 11:50
An alternative way is to expose a default state as object.__getstate__(). It is more efficient since it is implemented in C. Following patch implements this approach.
History
Date User Action Args
2020-08-14 06:25:45serhiy.storchakalinkissue41547 superseder
2017-11-15 21:15:23serhiy.storchakasetpull_requests: - pull_request4360
2017-11-15 21:14:27yselivanovsetpull_requests: + pull_request4360
2017-07-23 08:33:50serhiy.storchakasetnosy: + vstinner, r.david.murray

versions: + Python 3.7, - Python 3.6
2017-07-23 08:32:55serhiy.storchakasetpull_requests: + pull_request2873
2016-07-14 11:50:18serhiy.storchakasetfiles: + object_getstate.patch

messages: + msg270403
2016-06-18 19:59:45serhiy.storchakasetfiles: + copyreg_getstate2.patch

messages: + msg268830
2016-03-19 12:04:32berker.peksagsetnosy: + berker.peksag
2016-03-17 08:43:06serhiy.storchakacreate