➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Support out-of-band pickling for builtin types
Type: performance Stage:
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: jakirkham, pitrou
Priority: normal Keywords:

Created on 2020-05-21 19:25 by jakirkham, last changed 2022-04-11 14:59 by admin.

Messages (1)
msg369533 - (view) Author: (jakirkham) Date: 2020-05-21 19:25 
It would be nice (where possible) to support out-of-band pickling of builtin `bytes`-like types. This would allow binary data from these objects to be shipped along separately zero-copy and later reconstructed during unpickling.

It seems that `bytes`, `bytearray`, and `array` would be good candidates for this behavior. Not sure if `mmap` or `memoryview` would make sense as it might not be clear on how to reconstruct them during unpickling, but if someone sees a way those would be nice to support too.

To illustrate this a bit, here is the behavior with a `bytes` object today:

```
In [1]: import pickle

In [2]: b = b"abc"

In [3]: l = []

In [4]: p = pickle.dumps(b, protocol=5, buffer_callback=l.append)

In [5]: l
Out[5]: []
```

With this change, we would see this behavior instead:

```
In [1]: import pickle

In [2]: b = b"abc"

In [3]: l = []

In [4]: p = pickle.dumps(b, protocol=5, buffer_callback=l.append)

In [5]: l
Out[5]: [<pickle.PickleBuffer at 0x10efe7540>]
```

(This is my first Python bug submission. So apologies if I got turned around here. Please go easy on me :)
History
Date User Action Args
2022-04-11 14:59:31adminsetgithub: 84895
2021-09-30 00:30:58jakirkhamsetcomponents: + Library (Lib)
2021-09-27 19:08:47jakirkhamsetversions: + Python 3.10, Python 3.11
2020-05-22 04:33:39xtreaksetnosy: + pitrou
2020-05-21 19:25:54jakirkhamcreate