classification
Title: struct.unpack and cStringIO.StringIO don't support new buffer
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, benjamin.peterson, eric.araujo, kristjan.jonsson, lemburg, pitrou, python-dev, skrah, terry.reedy, tseaver
Priority: normal Keywords: needs review, patch

Created on 2010-10-27 10:36 by kristjan.jonsson, last changed 2013-03-22 22:53 by Arfrever. This issue is now closed.

Files
File name Uploaded Description Edit
newbuffer.patch kristjan.jonsson, 2010-10-27 10:36 review
Messages (11)
msg119687 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2010-10-27 10:36
When doing socket IO, it is beneficial to use a bytearra() and then using sock.recv_into() to avoid moving data about.
However, many useful functions still don't accept new style buffers, such as the bytearray and memoryview.  In particular, the struct module cannot unpack from them, and the StringIO doesn't accept them as input.

The attached patch adds new-buffer support to the struct module and cStringIO.
msg119689 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-27 11:13
You can't add buffer protocol support to cStringIO in a bugfix release, since it would be a new feature.
msg119750 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2010-10-28 01:46
I disagree.  It's not a new feature.  We're merely completing an old feature (adding new-style buffers from 3.x to 2.7) that wasn't fully implemented. by the core.  The new buffer isn't accepted in a lot of places where you'd expect it to be.

The good alternative, of course, would be to add this to "trunk" but for some reason that is frowned upon.

See also issue 10211
msg119781 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-10-28 13:52
MAL’s viewpoint from msg119721:

“The memoryview object was added to simplify porting applications to Python3. If that backport is incomplete in the sense that the memoryview object is not compatible with the standard Python2 object for such memory views, then I'd consider that a bug.

Without compatibility to the buffer objects, there's no way to make other Python2 buffer interface compatible object compatible to memoryviews.

Alternatively, the Python2 memoryview object implementation could also accept objects with the old buffer interface, much like "s*" does.

Note that the patch needs to check the buffer flags - writing to such buffers is not allowed.”
msg119939 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-10-29 22:03
In the absence of doc references in this and #10211 that would *clearly* settle the bug vs. feature issue, it strikes me as a bit murky. So I am inclined to agree with MAL that failure to achieve the stated, documented purpose is a bug. I strongly suspect that if these issues had been filed during the 2.7 beta phase, a fix would have been deemed permissible without much controversy.
msg156389 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2012-03-20 09:14
The patch is still there.  Any new consensus?
msg156426 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-03-20 16:08
Stefan, since you have been working on Memoryview, any opinions?
msg184524 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2013-03-18 21:19
We don't seem to be getting any traction on this.  2.7.4 is to be cut next weekend so it is becoming more urgent :)
msg184549 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-03-18 23:09
For the backport policy question, since there seems to be disagreement, consider a post on pydev asking for discussion and if no quick consensus, a ruling from Guido or designee.
msg184562 - (view) Author: Tres Seaver (tseaver) * Date: 2013-03-19 00:07
Patch looks good to me.  I would call it a bugfix, not a feature.
msg184717 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-03-20 00:40
New changeset be4bec689de3 by Kristján Valur Jónsson in branch '2.7':
Issue #10212: Support new buffer interface for struct.unpack and
http://hg.python.org/cpython/rev/be4bec689de3
History
Date User Action Args
2013-03-22 22:53:58Arfreversetstage: resolved
2013-03-22 17:25:56kristjan.jonssonsetstatus: open -> closed
resolution: fixed
2013-03-20 00:40:31python-devsetnosy: + python-dev
messages: + msg184717
2013-03-19 00:12:16Arfreversetnosy: + Arfrever
2013-03-19 00:07:09tseaversetnosy: + tseaver
messages: + msg184562
2013-03-18 23:09:19terry.reedysetmessages: + msg184549
2013-03-18 21:19:10kristjan.jonssonsetmessages: + msg184524
2012-03-20 16:08:51terry.reedysetnosy: + skrah
messages: + msg156426
2012-03-20 09:14:03kristjan.jonssonsetmessages: + msg156389
2010-10-29 22:03:35terry.reedysetnosy: + terry.reedy
messages: + msg119939
2010-10-28 13:53:01eric.araujosetnosy: + lemburg
2010-10-28 13:52:10eric.araujosetmessages: + msg119781
2010-10-28 01:46:52kristjan.jonssonsetstatus: pending -> open

messages: + msg119750
2010-10-27 11:23:55eric.araujosetstatus: open -> pending
nosy: + eric.araujo, benjamin.peterson
components: + Extension Modules, - Interpreter Core
2010-10-27 11:13:27pitrousetnosy: + pitrou
messages: + msg119689
2010-10-27 10:36:51kristjan.jonssoncreate