This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Added some explaining to pickle errors.
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.10
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Marco Sulla, serhiy.storchaka
Priority: normal Keywords:

Created on 2020-10-01 12:07 by Marco Sulla, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 22438 Marco Sulla, 2020-10-01 12:07
Messages (4)
msg377747 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2020-10-01 12:07
All pickle error messages in typeobject.c was a generic "cannot pickle 'type' object". Added some explaining for every individual error.
msg377763 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-10-01 15:25
It was intentionally left a generic "cannot pickle 'type' object". Because there are many ways to make an object pickleable, and it is non-pickleable in all other cases.

* "missing `__getstate__()` on variable-length object"

It creates false impression that implementing __getstate__() is the only way to solve the problem. Actually you can make that object pickleable by:

- implementing __reduce__() or __reduce_ex__()
- implementing __getnewargs__() or __getnewargs_ex__()
- implementing __getstate__()
- make it a subclass of list or dict

So more correct error message should contain "missing __reduce__(), __reduce_ex__(), __getnewargs__(), __getnewargs_ex__() and __getstate__() on variable-length object which is not a list or dict". And even this is not completely correct, because it has __reduce__() and __reduce_ex__() inherited from class object, but it is not enough. Completely correct error message can be hundreds characters long.

And this still may be misleading error message, because the root cause is not that some object is not pickleable, but that you try to pickle an object which has an unexpected non-pickleable component.

* "basic size greater than expected"

Basic size is an implementation detail, and the end user can have no guess what is it. Actually it means that an object implemented in C likely has some inner state and the pickle module does not have access to it because the object does not implement __reduce__(), __reduce_ex__(), __getnewargs__(), __getnewargs_ex__() or __getstate__().

* "object has no `__new__()`"

Every object has __new__ at Python level, even if its tp_new slot is NULL. tp_new is the C implementation detail and should not be mentioned in the error message. And the object without tp_new still can be pickleable, if it implements __reduce__() or __reduce_ex__().

It is all too complicated. It is better to omit details in error messages than provide not correct and direct user in wrong way.
msg377778 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2020-10-01 16:59
I do not remember the problem I had, but when I experimented with frozendict I get one of these errors. I failed to understand the problem so I added the additional info.

Maybe adding an assert in debug mode? It will be visible only to devs.
msg377815 - (view) Author: Marco Sulla (Marco Sulla) * Date: 2020-10-02 15:19
I closed it for this reason:
Date User Action Args
2022-04-11 14:59:36adminsetgithub: 86067
2020-10-02 15:19:55Marco Sullasetstatus: open -> closed

messages: + msg377815
stage: resolved
2020-10-01 16:59:20Marco Sullasetmessages: + msg377778
2020-10-01 15:25:42serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg377763
2020-10-01 12:07:50Marco Sullacreate