Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

struct.unpack and cStringIO.StringIO don't support new buffer #54421

Closed
kristjanvalur mannequin opened this issue Oct 27, 2010 · 11 comments
Closed

struct.unpack and cStringIO.StringIO don't support new buffer #54421

kristjanvalur mannequin opened this issue Oct 27, 2010 · 11 comments
Labels
extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error

Comments

@kristjanvalur
Copy link
Mannequin

kristjanvalur mannequin commented Oct 27, 2010

BPO 10212
Nosy @malemburg, @terryjreedy, @pitrou, @kristjanvalur, @benjaminp, @merwok, @skrah
Files
  • newbuffer.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2013-03-22.17:25:56.319>
created_at = <Date 2010-10-27.10:36:51.049>
labels = ['extension-modules', 'type-bug']
title = "struct.unpack and cStringIO.StringIO don't support new buffer"
updated_at = <Date 2013-03-22.22:53:58.754>
user = 'https://github.com/kristjanvalur'

    bugs.python.org fields:

    activity = <Date 2013-03-22.22:53:58.754>
actor = 'Arfrever'
assignee = 'none'
closed = True
closed_date = <Date 2013-03-22.17:25:56.319>
closer = 'kristjan.jonsson'
components = ['Extension Modules']
creation = <Date 2010-10-27.10:36:51.049>
creator = 'kristjan.jonsson'
dependencies = []
files = ['19382']
hgrepos = []
issue_num = 10212
keywords = ['patch', 'needs review']
message_count = 11.0
messages = ['119687', '119689', '119750', '119781', '119939', '156389', '156426', '184524', '184549', '184562', '184717']
nosy_count = 10.0
nosy_names = ['lemburg', 'terry.reedy', 'tseaver', 'pitrou', 'kristjan.jonsson', 'benjamin.peterson', 'eric.araujo', 'Arfrever', 'skrah', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue10212'
versions = ['Python 2.7']
    @kristjanvalur
    Copy link
    Mannequin Author

    kristjanvalur mannequin commented Oct 27, 2010

    When doing socket IO, it is beneficial to use a bytearra() and then using sock.recv_into() to avoid moving data about.
    However, many useful functions still don't accept new style buffers, such as the bytearray and memoryview. In particular, the struct module cannot unpack from them, and the StringIO doesn't accept them as input.

    The attached patch adds new-buffer support to the struct module and cStringIO.

    @kristjanvalur kristjanvalur mannequin added interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Oct 27, 2010
    @pitrou
    Copy link
    Member

    pitrou commented Oct 27, 2010

    You can't add buffer protocol support to cStringIO in a bugfix release, since it would be a new feature.

    @merwok merwok added extension-modules C modules in the Modules dir and removed interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Oct 27, 2010
    @kristjanvalur
    Copy link
    Mannequin Author

    kristjanvalur mannequin commented Oct 28, 2010

    I disagree. It's not a new feature. We're merely completing an old feature (adding new-style buffers from 3.x to 2.7) that wasn't fully implemented. by the core. The new buffer isn't accepted in a lot of places where you'd expect it to be.

    The good alternative, of course, would be to add this to "trunk" but for some reason that is frowned upon.

    See also bpo-10211

    @merwok
    Copy link
    Member

    merwok commented Oct 28, 2010

    MAL’s viewpoint from msg119721:

    “The memoryview object was added to simplify porting applications to Python3. If that backport is incomplete in the sense that the memoryview object is not compatible with the standard Python2 object for such memory views, then I'd consider that a bug.

    Without compatibility to the buffer objects, there's no way to make other Python2 buffer interface compatible object compatible to memoryviews.

    Alternatively, the Python2 memoryview object implementation could also accept objects with the old buffer interface, much like "s*" does.

    Note that the patch needs to check the buffer flags - writing to such buffers is not allowed.”

    @terryjreedy
    Copy link
    Member

    In the absence of doc references in this and bpo-10211 that would *clearly* settle the bug vs. feature issue, it strikes me as a bit murky. So I am inclined to agree with MAL that failure to achieve the stated, documented purpose is a bug. I strongly suspect that if these issues had been filed during the 2.7 beta phase, a fix would have been deemed permissible without much controversy.

    @kristjanvalur
    Copy link
    Mannequin Author

    kristjanvalur mannequin commented Mar 20, 2012

    The patch is still there. Any new consensus?

    @terryjreedy
    Copy link
    Member

    Stefan, since you have been working on Memoryview, any opinions?

    @kristjanvalur
    Copy link
    Mannequin Author

    kristjanvalur mannequin commented Mar 18, 2013

    We don't seem to be getting any traction on this. 2.7.4 is to be cut next weekend so it is becoming more urgent :)

    @terryjreedy
    Copy link
    Member

    For the backport policy question, since there seems to be disagreement, consider a post on pydev asking for discussion and if no quick consensus, a ruling from Guido or designee.

    @tseaver
    Copy link

    tseaver commented Mar 19, 2013

    Patch looks good to me. I would call it a bugfix, not a feature.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Mar 20, 2013

    New changeset be4bec689de3 by Kristján Valur Jónsson in branch '2.7':
    Issue bpo-10212: Support new buffer interface for struct.unpack and
    http://hg.python.org/cpython/rev/be4bec689de3

    @kristjanvalur kristjanvalur mannequin closed this as completed Mar 22, 2013
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @tseaver @merwok @pitrou @terryjreedy