classification
Title: StringIO.StringIO pickled in 2.7 is not unpickleable on 3.x
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.4, Python 3.5
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: Jelle Zijlstra, alexandre.vassalotti, pitrou, serhiy.storchaka
Priority: normal Keywords:

Created on 2014-11-17 07:41 by serhiy.storchaka, last changed 2016-06-26 21:59 by serhiy.storchaka. This issue is now closed.

Messages (5)
msg231268 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-11-17 07:41
StringIO.StringIO is pickleable and unpickleable on 2.7 but is not unpickleable on 3.x.

Python 2.7:
>>> import pickle, StringIO
>>> pickle.dumps(StringIO.StringIO('abc'), 2)
'\x80\x02(cStringIO\nStringIO\nq\x00oq\x01}q\x02(U\tsoftspaceq\x03K\x00U\x07buflistq\x04]q\x05U\x03posq\x06K\x00U\x03lenq\x07K\x03U\x06closedq\x08\x89U\x03bufq\tU\x03abcq\nub.'
>>> pickle.loads(b'\x80\x02(cStringIO\nStringIO\nq\x00oq\x01}q\x02(U\tsoftspaceq\x03K\x00U\x07buflistq\x04]q\x05U\x03posq\x06K\x00U\x03lenq\x07K\x03U\x06closedq\x08\x89U\x03bufq\tU\x03abcq\nub.')
<StringIO.StringIO instance at 0xb70c122c>

On 3.x I got an error:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: _io.StringIO.__setstate__ argument should be 4-tuple, got dict
msg236784 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2015-02-27 16:41
Seems fine on Windows 8.1

c:\Users\Mark\Documents\MyPython>c:\cpython\PCbuild\amd64\python.exe
Python 3.5.0a1+ (default:344d57c521b9+, Feb 27 2015, 13:39:56) [MSC v.1800 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle, io
>>> pickle.dumps(io.StringIO('abc'), 2)
b'\x80\x02c_io\nStringIO\nq\x00)\x81q\x01(X\x03\x00\x00\x00abcq\x02X\x01\x00\x00\x00\nq\x03K\x00Ntq\x04b.'
>>> pickle.loads(b'\x80\x02c_io\nStringIO\nq\x00)\x81q\x01(X\x03\x00\x00\x00abcq\x02X\x01\x00\x00\x00\nq\x03K\x00Ntq\x04b.')
<_io.StringIO object at 0x0000004C6E604288>
>>>
msg236799 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-27 17:20
The issue is about StringIO.StringIO, not io.StringIO.
msg267016 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * (Python triager) Date: 2016-06-03 01:12
Unless I'm missing something, StringIO doesn't exist as a module in Python 3, as mentioned in https://docs.python.org/3.0/whatsnew/3.0.html#text-vs-data-instead-of-unicode-vs-8-bit.

If you're trying to unpickle in 3.x an object pickled by 2.7.x, I'm not sure there is a guarantee that unpickling works for objects in modules that were renamed between 2.x and 3.x. This also doesn't work if you pickle a ConfigParser in 2.7 and unpickle it in 3.x.
msg269320 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-06-26 21:59
There is the _compat_pickle module that purposed to support backward and forward compatibility for renamed or moved classes and modules.

StringIO.StringIO in 2.x corresponds to io.StringIO or io.BytesIO. The problem is that we can't determine what of classes is a match in every concrete case, especially if StringIO.StringIO is empty.

But since cStringIO.StringIO is not pickleable either, I now think that it is not worth to support to support compatibility with pickled StringIO.StringIO instances.
History
Date User Action Args
2016-06-26 21:59:59serhiy.storchakasetstatus: open -> closed
resolution: wont fix
messages: + msg269320

stage: resolved
2016-06-03 01:15:58BreamoreBoysetnosy: - BreamoreBoy
2016-06-03 01:12:54Jelle Zijlstrasetnosy: + Jelle Zijlstra
messages: + msg267016
2015-02-27 17:20:16serhiy.storchakasetmessages: + msg236799
2015-02-27 16:41:55BreamoreBoysetnosy: + BreamoreBoy
messages: + msg236784
2014-11-17 07:41:25serhiy.storchakacreate