This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Cannot pickle Ellipsis or NotImplemented
Type: enhancement Stage: resolved
Components: Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: lukasz.langa Nosy List: James.Sanders, alexandre.vassalotti, hynek, lukasz.langa, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2012-01-23 19:24 by James.Sanders, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
pickling_ellipsis_notimplemented.patch James.Sanders, 2012-01-24 14:33 review
Messages (10)
msg151838 - (view) Author: James Sanders (James.Sanders) Date: 2012-01-23 19:24
At present, the built-in constants Ellipsis (...) and NotImplemented cannot be pickled.  Perhaps there is a good reason for this, but the only discussion I can find is at msg<108957>, where it is stated that these values (along with their types, and type(None)) cannot be pickled.

I ran across this in a class that keeps track of numpy-style slicing operations, and so sometimes stores references to Ellipsis.  While this is easy to work around, it does seem a little surprising that ... cannot be pickled, when slice objects can be.  I don't know if there is a likely use for pickling NotImplemented.

If this is not changed, perhaps it could be explicitly stated in the documentation that these objects cannot be pickled?
msg151843 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-01-23 21:05
I think it's a reasonable feature request. Now someone has to write a patch for it.
msg151844 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2012-01-23 21:18
We will need to bump the protocol number to add support for None, Ellipsis, and NotImplemented. Antoine, can you add this to PEP 3154?
msg151852 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-01-23 21:56
> We will need to bump the protocol number to add support for None,
> Ellipsis, and NotImplemented. Antoine, can you add this to PEP 3154?

I don't think this needs a protocol bump. These are global singletons,
they can be pickled as such, and they will be unpickleable on any
Python.
msg151911 - (view) Author: James Sanders (James.Sanders) Date: 2012-01-24 14:33
I've submitted a patch that just uses save_global to pickle Ellipsis and NotImplemented, so the resulting pickle should be unpicklable anywhere.  I'm completely new to the C API so not sure if the way I am building python strings (to pass to save_global) is correct, but it seems to work.
msg153708 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-02-19 16:09
I haven't taken a look at the patch in detail, but it lacks some tests.
Pickling tests are generally in Lib/test/pickletester.py, they are invoked by test_pickle and test_pickletools.
msg153709 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-02-19 16:18
> I'm completely new to the C API so not sure if the way I am building
> python strings (to pass to save_global) is correct

It's correct but it's probably less efficient than calling a more specialized function, such as PyUnicode_FromString():
http://docs.python.org/dev/c-api/unicode.html#PyUnicode_FromString

You also have to check the return value (for non-NULL, NULL meaning an error occurred) before calling save_global with it.
msg155460 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2012-03-12 18:45
There is indeed a 5% performance gain from using PyUnicode_FromString instead of the generic Py_BuildValue function.
msg155461 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-03-12 19:00
New changeset 5832d3622473 by Łukasz Langa in branch 'default':
Fixes #13842: cannot pickle Ellipsis or NotImplemented.
http://hg.python.org/cpython/rev/5832d3622473
msg155492 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-03-12 21:59
New changeset 5353357382e2 by Łukasz Langa in branch 'default':
#13842: check whether PyUnicode_FromString succeeded
http://hg.python.org/cpython/rev/5353357382e2
History
Date User Action Args
2022-04-11 14:57:26adminsetgithub: 58050
2012-03-12 21:59:55python-devsetmessages: + msg155492
2012-03-12 19:01:04lukasz.langasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2012-03-12 19:00:05python-devsetnosy: + python-dev
messages: + msg155461
2012-03-12 18:45:33lukasz.langasetmessages: + msg155460
2012-03-08 22:01:14lukasz.langasetassignee: lukasz.langa

nosy: + lukasz.langa
2012-02-19 16:18:17pitrousetmessages: + msg153709
2012-02-19 16:09:37pitrousetnosy: + hynek
2012-02-19 16:09:24pitrousetmessages: + msg153708
stage: needs patch -> patch review
2012-01-24 14:33:51James.Sanderssetfiles: + pickling_ellipsis_notimplemented.patch
keywords: + patch
messages: + msg151911
2012-01-23 21:56:30pitrousetmessages: + msg151852
2012-01-23 21:18:17alexandre.vassalottisetmessages: + msg151844
2012-01-23 21:05:58pitrousetmessages: + msg151843
stage: needs patch
2012-01-23 19:24:18James.Sanderscreate