classification
Title: fileinput, StringIO, and cStringIO do not support the with protocol
Type: behavior Stage: patch review
Components: Extension Modules, Library (Lib) Versions: Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: alexandre.vassalotti, belopolsky, brian.curtin, georg.brandl, ygale (5)
Priority: low Keywords easy, patch

Created on 2007-10-16 11:59 by ygale, last changed 2009-11-17 21:03 by gvanrossum.

Files
File name Uploaded Description Edit Remove
with_StringIO.diff belopolsky, 2008-02-20 17:07 diff against revision 60906
issue1286.patch brian.curtin, 2009-11-17 20:59 patch implementing context manager for fileinput
Messages (10)
msg56497 - (view) Author: Yitz Gale (ygale) Date: 2007-10-16 12:42
The standard idiom for opening a file is now with open...
So I think it should be a goal that this should work with
any built-in file-like object that needs to be closed,
without having to explicitly wrap it in closing().
It certainly should work for fileinput and StringIO - since
these really are files, in some sense.
msg56636 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) Date: 2007-10-22 01:16
Do you have a use-case for this? In Py3k, I don't think adding support
for the 'with' statement to StringIO makes any sense, since the close()
method does nothing.
msg56643 - (view) Author: Yitz Gale (ygale) Date: 2007-10-22 08:05
These objects are supposed to be drop-in replacements
for file handles. Except in legacy code, the way you
use a file handle is:

with function_to_create_fh as fh:
  <do stuff>

If these objects do not support the with protocol,
the only way to use them is to refactor this code.
That may not even be possible, e.g., if it is in
a library, or it may not be desirable to refactor.

Even if you can refactor it, I don't think you
can call these objects file-like objects if
you can't use them like a file.

Note that in legacy code, you used to write:

fh = function_to_get_fh
try:
  <do stuff>
finally:
  fh.close()

If function_to_get_fh happens to return some other
file-like object and not an actual file,
this still works fine without any refactoring.

You wrote:
> In Py3k, I don't think adding support
> for the 'with' statement to StringIO makes
> any sense, since the close()
> method does nothing.

So do you propse removing the close() method
altogether? Of course the answer is "no",
because then you could not use the object like
a file. The same is true for the with protocol.

(I now retract the words "that needs to be closed"
from my original comment. All file-like objects
should support the with protocol, regardles of
whether their close() method actually does anything.)
msg56678 - (view) Author: Georg Brandl (georg.brandl) Date: 2007-10-23 12:31
Makes sense to me.
msg56714 - (view) Author: Guido van Rossum (gvanrossum) Date: 2007-10-24 18:30
Let's not get overexcited. I agree that it makes sense for fileinput,
but I disagree about *StringIO; its close() isn't needed to free
resources (since it doesn't use up a scarce resource like a file
descriptor).

Can you whip up a patch for fileinput?  Please include unit tests and
documentation.
msg56740 - (view) Author: Yitz Gale (ygale) Date: 2007-10-25 12:08
I was actually bitten badly by this issue with
StringIO. I added fileinput only as an afterthought.

In an xml.sax app, I needed seek() support for a
codec-wrapped file handle, so I over-wrapped it with
StringIO. The result was that I had to refactor code all over
the place to handle StringIO as a special case. What a
mess!

Why is this getting over-excited? It's a very
lightweight change. You can't beat the cost/benefit ratio.
msg56757 - (view) Author: Guido van Rossum (gvanrossum) Date: 2007-10-25 22:48
2007/10/25, Yitz Gale <report@bugs.python.org>:
> I was actually bitten badly by this issue with
> StringIO. I added fileinput only as an afterthought.
>
> In an xml.sax app, I needed seek() support for a
> codec-wrapped file handle, so I over-wrapped it with
> StringIO. The result was that I had to refactor code all over
> the place to handle StringIO as a special case. What a
> mess!

I don't understand. What did your code look like after the refactoring?

I find that typically a useful idiom is to have one piece of code
handle opening/closing of streams and let the rest of the code just
deal with streams without ever closing them. E.g.

f = open(filename)
try:
  process(f)
finally:
  f.close()

or, if you want:

with open(filename) as f:
  process(f)

As I don't understand how you are working the StringIO() call into
this I'm still not sure what the issue is.

> Why is this getting over-excited? It's a very
> lightweight change. You can't beat the cost/benefit ratio.

Until you submit a patch it's more work for me. :-)
msg59778 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) Date: 2008-01-12 02:54
FYI, StringIO and BytesIO, in Python 3K, already support the context
management protocol.
msg62595 - (view) Author: Alexander Belopolsky (belopolsky) Date: 2008-02-20 17:07
Attached patch implements context management protocol for StringIO.
msg95401 - (view) Author: Brian Curtin (brian.curtin) Date: 2009-11-17 20:59
Attached is a patch against trunk r76325 which implements context
manager support for fileinput/FileInput, with tests and doc change.
History
Date User Action Args
2009-11-17 21:03:15gvanrossumsetnosy: - gvanrossum
2009-11-17 20:59:29brian.curtinsetfiles: + issue1286.patch

nosy: + brian.curtin
messages: + msg95401

keywords: + patch
2009-05-15 01:50:11ajaksu2setstage: patch review
2008-10-04 15:49:31benjamin.petersonsetversions: + Python 2.7, - Python 2.6
2008-02-20 17:07:09belopolskysetfiles: + with_StringIO.diff
nosy: + belopolsky
messages: + msg62595
components: + Library (Lib)
versions: - Python 3.0
2008-01-12 02:54:01alexandre.vassalottisetmessages: + msg59778
2008-01-12 01:27:28akuchlingsetkeywords: + easy
2007-10-25 22:48:59gvanrossumsetmessages: + msg56757
2007-10-25 12:09:00ygalesetmessages: + msg56740
2007-10-24 18:30:38gvanrossumsetpriority: low
nosy: + gvanrossum
messages: + msg56714
versions: - Python 2.5
2007-10-23 12:31:42georg.brandlsetnosy: + georg.brandl
messages: + msg56678
2007-10-22 08:05:51ygalesetmessages: + msg56643
2007-10-22 01:16:28alexandre.vassalottisetnosy: + alexandre.vassalotti
messages: + msg56636
2007-10-16 12:42:36ygalesetmessages: + msg56497
2007-10-16 11:59:57ygalecreate