Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BytesIO and StringIO values unavailable when closed #67288

Closed
vadmium opened this issue Dec 22, 2014 · 12 comments
Closed

BytesIO and StringIO values unavailable when closed #67288

vadmium opened this issue Dec 22, 2014 · 12 comments
Assignees
Labels
topic-IO type-bug An unexpected behavior, bug, or error

Comments

@vadmium
Copy link
Member

vadmium commented Dec 22, 2014

BPO 23099
Nosy @pitrou, @vadmium, @serhiy-storchaka
Files
  • bytesio_exported_reject_close.patch
  • bytesio_exported_reject_close.v2.patch
  • bytesio_exported_reject_close.v3.patch
  • bytesio_exported_reject_close.v4.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
    closed_at = <Date 2015-02-03.07:40:55.918>
    created_at = <Date 2014-12-22.11:41:17.260>
    labels = ['type-bug', 'expert-IO']
    title = 'BytesIO and StringIO values unavailable when closed'
    updated_at = <Date 2015-02-03.07:40:55.917>
    user = 'https://github.com/vadmium'

    bugs.python.org fields:

    activity = <Date 2015-02-03.07:40:55.917>
    actor = 'serhiy.storchaka'
    assignee = 'serhiy.storchaka'
    closed = True
    closed_date = <Date 2015-02-03.07:40:55.918>
    closer = 'serhiy.storchaka'
    components = ['IO']
    creation = <Date 2014-12-22.11:41:17.260>
    creator = 'martin.panter'
    dependencies = []
    files = ['37531', '37611', '37942', '37973']
    hgrepos = []
    issue_num = 23099
    keywords = ['patch']
    message_count = 12.0
    messages = ['233016', '233018', '233023', '233036', '233506', '235108', '235138', '235159', '235222', '235229', '235304', '235317']
    nosy_count = 4.0
    nosy_names = ['pitrou', 'python-dev', 'martin.panter', 'serhiy.storchaka']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue23099'
    versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

    @vadmium
    Copy link
    Member Author

    vadmium commented Dec 22, 2014

    IOBase.close() doc says file operations raise ValueError, but it is not obvious to me that reading back the “file” buffer is a file operation.

    >>> with BytesIO() as b:
    ...     b.write(b"123")
    ... 
    3
    >>> b.getvalue()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: I/O operation on closed file.

    Even worse, the memoryview gets corrupted on close():

    >>> b = BytesIO(b"123")
    >>> m = b.getbuffer()
    >>> b.close()
    >>> bytes(m)
    b'\x98\x02>'

    I also noticed that in the “io” implementation, writing to the file seems to be completely disallowed, even if it would not seem to change the size:

    >>> b = BytesIO(b"123")
    >>> m = b.getbuffer()
    >>> b.write(b"x")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    BufferError: Existing exports of data: object cannot be re-sized

    @vadmium vadmium added topic-IO type-bug An unexpected behavior, bug, or error labels Dec 22, 2014
    @serhiy-storchaka
    Copy link
    Member

    getvalue() doesn't work after close() for purpose. close() frees memory used by BytesIO.

    >>> import io, sys
    >>> bio = io.BytesIO()
    >>> sys.getsizeof(bio)
    52
    >>> bio.write(b'x'*1000)
    1000
    >>> sys.getsizeof(bio)
    1053
    >>> bio.close()
    >>> sys.getsizeof(bio)
    52

    Changing the behavior will cause regression.

    The behavior of memoryview looks as a bug.

    @pitrou
    Copy link
    Member

    pitrou commented Dec 22, 2014

    Even worse, the memoryview gets corrupted on close():

    The BytesIO object should probably reject closing when a buffer is exported.

    writing to the file seems to be completely disallowed, even if it
    would not seem to change the size:

    An enhancement is probably possible there.

    @serhiy-storchaka
    Copy link
    Member

    Here is a patch which rejects close() when a buffer is exported.

    @vadmium
    Copy link
    Member Author

    vadmium commented Jan 6, 2015

    Updated patch, to also document the BytesIO buffer is no longer available when closed. The StringIO documentation actually already says this, but I rarely use StringIO. :)

    @serhiy-storchaka
    Copy link
    Member

    Why not just copy the StringIO documentation?

    @vadmium
    Copy link
    Member Author

    vadmium commented Feb 1, 2015

    I can live with the wording of StringIO, but personally prefer my v2 patch. I now understand that calling close() for Bytes and StringIO objects is intended to immediately free the memory buffer holding the file data (like deleting a file in Windows). So I think it would be better to document this as a property of the whole object, rather than a special exception for each of the getbuffer(), getvalue() non-file-API methods.

    I’m happy to propose a similar wording for the StringIO class if you want to make them consistent.

    @serhiy-storchaka
    Copy link
    Member

    Yes, it would be good to make the documentation of BytesIO and StringIO consistent.

    @vadmium
    Copy link
    Member Author

    vadmium commented Feb 2, 2015

    Here is an option that moves the documentation for discarding the buffer into the class description for both BytesIO and StringIO; what do you think? I would be happy enough with any of the last three patches, so I don’t want to hold this up forever :)

    @serhiy-storchaka
    Copy link
    Member

    LGTM.

    @serhiy-storchaka serhiy-storchaka self-assigned this Feb 2, 2015
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Feb 3, 2015

    New changeset e62d54128bd3 by Serhiy Storchaka in branch '3.4':
    Issue bpo-23099: Closing io.BytesIO with exported buffer is rejected now to
    https://hg.python.org/cpython/rev/e62d54128bd3

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Feb 3, 2015

    New changeset b9d4c013b09a by Serhiy Storchaka in branch 'default':
    Issue bpo-23099: Closing io.BytesIO with exported buffer is rejected now to
    https://hg.python.org/cpython/rev/b9d4c013b09a

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    topic-IO type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants