classification
Title: BytesIO and StringIO values unavailable when closed
Type: behavior Stage: resolved
Components: IO Versions: Python 3.5, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: martin.panter, pitrou, python-dev, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2014-12-22 11:41 by martin.panter, last changed 2015-02-03 07:40 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
bytesio_exported_reject_close.patch serhiy.storchaka, 2014-12-23 07:26 review
bytesio_exported_reject_close.v2.patch martin.panter, 2015-01-06 07:01 review
bytesio_exported_reject_close.v3.patch serhiy.storchaka, 2015-01-31 13:12 review
bytesio_exported_reject_close.v4.patch martin.panter, 2015-02-02 03:24 review
Messages (12)
msg233016 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2014-12-22 11:41
IOBase.close() doc says file operations raise ValueError, but it is not obvious to me that reading back the “file” buffer is a file operation.

>>> with BytesIO() as b:
...     b.write(b"123")
... 
3
>>> b.getvalue()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: I/O operation on closed file.

Even worse, the memoryview gets corrupted on close():

>>> b = BytesIO(b"123")
>>> m = b.getbuffer()
>>> b.close()
>>> bytes(m)
b'\x98\x02>'

I also noticed that in the “io” implementation, writing to the file seems to be completely disallowed, even if it would not seem to change the size:

>>> b = BytesIO(b"123")
>>> m = b.getbuffer()
>>> b.write(b"x")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
BufferError: Existing exports of data: object cannot be re-sized
msg233018 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-12-22 12:29
getvalue() doesn't work after close() for purpose. close() frees memory used by BytesIO.

>>> import io, sys
>>> bio = io.BytesIO()
>>> sys.getsizeof(bio)
52
>>> bio.write(b'x'*1000)
1000
>>> sys.getsizeof(bio)
1053
>>> bio.close()
>>> sys.getsizeof(bio)
52

Changing the behavior will cause regression.

The behavior of memoryview looks as a bug.
msg233023 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-12-22 19:46
> Even worse, the memoryview gets corrupted on close():

The BytesIO object should probably reject closing when a buffer is exported.

> writing to the file seems to be completely disallowed, even if it 
> would not seem to change the size:

An enhancement is probably possible there.
msg233036 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-12-23 07:26
Here is a patch which rejects close() when a buffer is exported.
msg233506 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-01-06 07:01
Updated patch, to also document the BytesIO buffer is no longer available when closed. The StringIO documentation actually already says this, but I rarely use StringIO. :)
msg235108 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-01-31 13:12
Why not just copy the StringIO documentation?
msg235138 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-02-01 00:57
I can live with the wording of StringIO, but personally prefer my v2 patch. I now understand that calling close() for Bytes and StringIO objects is intended to immediately free the memory buffer holding the file data (like deleting a file in Windows). So I think it would be better to document this as a property of the whole object, rather than a special exception for each of the getbuffer(), getvalue() non-file-API methods.

I’m happy to propose a similar wording for the StringIO class if you want to make them consistent.
msg235159 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-01 10:53
Yes, it would be good to make the documentation of BytesIO and StringIO consistent.
msg235222 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-02-02 03:24
Here is an option that moves the documentation for discarding the buffer into the class description for both BytesIO and StringIO; what do you think? I would be happy enough with any of the last three patches, so I don’t want to hold this up forever :)
msg235229 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-02 07:32
LGTM.
msg235304 - (view) Author: Roundup Robot (python-dev) Date: 2015-02-03 00:05
New changeset e62d54128bd3 by Serhiy Storchaka in branch '3.4':
Issue #23099: Closing io.BytesIO with exported buffer is rejected now to
https://hg.python.org/cpython/rev/e62d54128bd3
msg235317 - (view) Author: Roundup Robot (python-dev) Date: 2015-02-03 07:31
New changeset b9d4c013b09a by Serhiy Storchaka in branch 'default':
Issue #23099: Closing io.BytesIO with exported buffer is rejected now to
https://hg.python.org/cpython/rev/b9d4c013b09a
History
Date User Action Args
2015-02-03 07:40:55serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2015-02-03 07:31:49python-devsetmessages: + msg235317
2015-02-03 00:05:21python-devsetnosy: + python-dev
messages: + msg235304
2015-02-02 07:32:50serhiy.storchakasetassignee: serhiy.storchaka
messages: + msg235229
2015-02-02 03:24:33martin.pantersetfiles: + bytesio_exported_reject_close.v4.patch

messages: + msg235222
2015-02-01 10:53:05serhiy.storchakasetmessages: + msg235159
2015-02-01 00:57:25martin.pantersetmessages: + msg235138
2015-01-31 13:12:03serhiy.storchakasetfiles: + bytesio_exported_reject_close.v3.patch

messages: + msg235108
2015-01-06 07:01:24martin.pantersetfiles: + bytesio_exported_reject_close.v2.patch

messages: + msg233506
2014-12-23 07:26:17serhiy.storchakasetfiles: + bytesio_exported_reject_close.patch
versions: + Python 2.7, Python 3.5
messages: + msg233036

keywords: + patch
stage: patch review
2014-12-22 19:46:49pitrousetnosy: + pitrou
messages: + msg233023
2014-12-22 12:29:01serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg233018
2014-12-22 11:41:17martin.pantercreate