This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: __setstate__ is called for false values
Type: behavior Stage: patch review
Components: Documentation Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: alexandre.vassalotti, belopolsky, docs@python, eltoder, georg.brandl, iritkatriel, pitrou, r.david.murray, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2011-06-09 02:20 by eltoder, last changed 2022-04-11 14:57 by admin.

Files
File name Uploaded Description Edit
setstate.diff eltoder, 2011-06-16 02:57
Messages (6)
msg137935 - (view) Author: Eugene Toder (eltoder) * Date: 2011-06-09 02:20
Pickle documentation [1] says:

""" Note: If __getstate__() returns a false value, the __setstate__() method will not be called upon unpickling. """

However, this isn't quite true. This depends on the version of pickle protocol. A small example:

>>> class Pockle(object):
	def __getstate__(self):
		return 0
	def __setstate__(self, state):
		sys.stdout.write('__setstate__ is called!\n')

>>> for p in range(4):
	sys.stdout.write('protocol %d: ' % p)
	pickle.loads(pickle.dumps(Pockle(), p))

protocol 0: <__main__.Pockle object at 0x0000000002EAE3C8>
protocol 1: <__main__.Pockle object at 0x0000000002EAE358>
protocol 2: __setstate__ is called!
<__main__.Pockle object at 0x0000000002EAE3C8>
protocol 3: __setstate__ is called!
<__main__.Pockle object at 0x0000000002EAE358>

So for protocols >= 2 setstate is called. This is caused by object.__reduce_ex__ returning different tuples for different protocol versions:

>>> for p in range(4):
	sys.stdout.write('protocol %d: %s\n' % (p, Pockle().__reduce_ex__(p)))

	
protocol 0: (<function _reconstructor at 0x0000000001F03048>, (<class '__main__.Pockle'>, <class 'object'>, None))
protocol 1: (<function _reconstructor at 0x0000000001F03048>, (<class '__main__.Pockle'>, <class 'object'>, None))
protocol 2: (<function __newobj__ at 0x0000000001F03148>, (<class '__main__.Pockle'>,), 0, None, None)
protocol 3: (<function __newobj__ at 0x0000000001F03148>, (<class '__main__.Pockle'>,), 0, None, None)

Implementation of reduce_ex for protos 0-1 in copy_reg.py contains the documented check: http://hg.python.org/cpython/file/f1509fc75435/Lib/copy_reg.py#l85

Implementation for proto 2+ in typeobject.c is happy with any value: http://hg.python.org/cpython/file/f1509fc75435/Objects/typeobject.c#l3205

Pickle itself only ignores None, not any false value: http://hg.python.org/cpython/file/f1509fc75435/Lib/pickle.py#l418

I think this is a documentation issue at this point.

[1] http://docs.python.org/py3k/library/pickle.html#pickle.object.__setstate__
msg137936 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-06-09 02:37
See also #6827, just for some background on the current docs.
msg138410 - (view) Author: Eugene Toder (eltoder) * Date: 2011-06-16 02:57
So how about this correction?
msg138573 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-06-18 08:26
Well, this looks correct then.
msg265087 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-07 19:02
Actually this wording is not quite correct. __setstate__() is called for any pickled state. It is not called only if the state is not pickled. The state is not pickled if reducing method (__reduce_ex__ or __reduce__) doesn't return state or returns None as a state. Default reducing method for protocol 0 and 1 doesn't return a state if __getstate__() returns false value.

There are many other details of pickle protocol that are not correctly documented in the documentation of the pickle module. PEP 307 documents pickle protocol more correctly.
msg407755 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-05 23:10
See also issue26695.
History
Date User Action Args
2022-04-11 14:57:18adminsetgithub: 56499
2021-12-05 23:10:56iritkatrielsetversions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.5, Python 3.6
nosy: + iritkatriel

messages: + msg407755

type: behavior
2016-08-10 17:32:01matrixisesetversions: + Python 3.5, Python 3.6, - Python 2.7, Python 3.3, Python 3.4
2016-05-07 19:02:41serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg265087
2013-12-07 09:33:40alexandre.vassalottisetversions: + Python 2.7, Python 3.3
2013-12-07 09:32:30alexandre.vassalottisetversions: + Python 3.4, - Python 2.6, Python 3.1, Python 2.7, Python 3.2, Python 3.3
nosy: + docs@python

assignee: docs@python
components: + Documentation, - Library (Lib)
stage: patch review
2011-06-18 08:26:22georg.brandlsetmessages: + msg138573
2011-06-16 02:57:48eltodersetfiles: + setstate.diff

nosy: + georg.brandl, belopolsky
messages: + msg138410

keywords: + patch
2011-06-09 02:59:01eltodersetnosy: + pitrou, alexandre.vassalotti
2011-06-09 02:37:22r.david.murraysetnosy: + r.david.murray
messages: + msg137936
2011-06-09 02:20:57eltodercreate