classification
Title: File objects should not pickleable
Type: behavior Stage: resolved
Components: IO, Library (Lib) Versions: Python 3.2
process
Status: closed Resolution: fixed
Dependencies: 10173 Superseder:
Assigned To: Nosy List: alex, alexandre.vassalotti, amaury.forgeotdarc, benjamin.peterson, georg.brandl, pitrou
Priority: normal Keywords: patch

Created on 2010-10-23 17:19 by pitrou, last changed 2010-11-05 21:16 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
pickleio.patch pitrou, 2010-10-23 18:10
Messages (9)
msg119450 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-23 17:19
In Python 3, pickle accepts to serialize a file object, but the result is nonsensical when unpickled. I think we should explicitly forbid pickling of FileIO, Buffered{Reader,Writer} and TextIOWrapper objects.
msg119457 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2010-10-23 17:50
Sounds like a good idea to me.
msg119459 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-23 18:10
Here is a patch, but issue10173 must probably be fixed first.
msg119470 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-10-23 19:37
The patch modifies _io.TextIOWrapper, but not _pyio.TextIOWrapper. Is there a reason?
msg119471 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-23 19:41
> The patch modifies _io.TextIOWrapper, but not _pyio.TextIOWrapper. Is
> there a reason?

Yes, two of them:
- modifying _pyio.Buffered* is enough to trigger the TypeError
- _pyio.StringIO inherits from TextIOWrapper, and it must be pickleable
msg119473 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2010-10-23 20:58
I don't see why Buffered or TextIO's shouldn't be pickleable, ISTM their pickleability should be based on what the underlying file obj is.
msg119474 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2010-10-23 21:09
What would be the use case for that?
msg119475 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-23 21:11
> I don't see why Buffered or TextIO's shouldn't be pickleable, ISTM
> their pickleability should be based on what the underlying file obj
> is.

That could be. Right now, though, pickling them gives nonsensical
results and I think it would be better to give an explicit error
message.

(a problem with making them pickleable is that, by exposing details of
the internal structure in the pickle, you're limiting what
implementation changes you can do in the future)
msg120536 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-11-05 21:16
Committed in r86220. I won't backport it since it would risk breaking existing code, although relying on this is really a bug in itself.
History
Date User Action Args
2010-11-05 21:16:14pitrousetstatus: open -> closed
resolution: fixed
messages: + msg120536

stage: resolved
2010-10-23 21:11:22pitrousetmessages: + msg119475
2010-10-23 21:09:05georg.brandlsetmessages: + msg119474
2010-10-23 20:58:29alexsetnosy: + alex
messages: + msg119473
2010-10-23 19:41:45pitrousetmessages: + msg119471
2010-10-23 19:37:35amaury.forgeotdarcsetmessages: + msg119470
2010-10-23 18:10:51pitrousetfiles: + pickleio.patch
keywords: + patch
dependencies: + Don't pickle TestCase instances in test_multiprocessing
messages: + msg119459
2010-10-23 17:50:25georg.brandlsetnosy: + georg.brandl
messages: + msg119457
2010-10-23 17:19:26pitroucreate