Author serhiy.storchaka
Recipients alexandre.vassalotti, belopolsky, eddygeek, pitrou, serhiy.storchaka, tanzer@swing.co.at, tim.peters
Date 2015-10-12.18:07:56
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1444673276.99.0.248226167349.issue22005@psf.upfronthosting.co.za>
In-reply-to
Content
There are two issues here.

1. datetime.datetime accepts only bytes, not str.
2. Unpickling non-ASCII str pickled in Python 2 raises an error by default.

The second issue usually hides the first one. The demonstration of the first issue:

>>> pickle.loads(b'cdatetime\ndatetime\n(U\n\x07l\x01\x01\x00\x00\x00\x00\x00\x00tR.')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: an integer is required (got type str)

The first issue can be solved by accepting str argument and encoding it to bytes. The second issue can be solved by changing an encoding or an error handler. Following patch uses the "surrogateescape" error handler.

>>> pickle.loads(b'cdatetime\ndatetime\n(U\n\x07l\x01\x01\x00\x00\x00\x00\xc3\xa4tR.')
datetime.datetime(1900, 1, 1, 0, 0, 0, 50084)

Unfortunately setting the "surrogateescape" error handler by default has a side effect. It can hide string decoding errors. In addition, unpickling datetime will not always work with different encodings.

>>> pickle.loads(b'cdatetime\ndatetime\n(U\n\x07l\x01\x01\x00\x00\x00\x00\xc3\xa4tR.', encoding='latin1')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode characters in position 8-9: ordinal not in range(128)
>>> pickle.loads(b'cdatetime\ndatetime\n(U\n\x07l\x01\x01\x00\x00\x00\x00\xc3\xa4tR.', encoding='utf-8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: an integer is required (got type str)
History
Date User Action Args
2015-10-12 18:07:57serhiy.storchakasetrecipients: + serhiy.storchaka, tim.peters, belopolsky, pitrou, alexandre.vassalotti, eddygeek, tanzer@swing.co.at
2015-10-12 18:07:56serhiy.storchakasetmessageid: <1444673276.99.0.248226167349.issue22005@psf.upfronthosting.co.za>
2015-10-12 18:07:56serhiy.storchakalinkissue22005 messages
2015-10-12 18:07:56serhiy.storchakacreate