classification
Title: memoryview doesn't allow tuple-indexing
Type: enhancement Stage: resolved
Components: Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: pitrou, python-dev, r.david.murray, serhiy.storchaka, skrah
Priority: normal Keywords: patch

Created on 2015-03-10 21:46 by pitrou, last changed 2015-03-19 23:11 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
memoryview_tuple_indexing.patch pitrou, 2015-03-10 22:43 review
memoryview_tuple_indexing2.patch pitrou, 2015-03-18 22:53 review
Messages (18)
msg237814 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-03-10 21:46
It is a bit of pity. A least non-sliced indexing should be able to work:

>>> import numpy as np
>>> a = np.arange(10)
>>> memoryview(a)[0]
0
>>> a = np.arange(10).reshape((2,5))
>>> a[0,1]
1
>>> memoryview(a)[0,1]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: memoryview: invalid slice key
msg237820 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-03-10 22:43
Here is a patch.
msg237926 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-03-12 12:52
Aren't there wider implications of python starting to support tuple indexing?  If we make this work, aren't people going to expect [[1,2], [3,4]][0,1] to work?
msg237927 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-03-12 12:57
> If we make this work, aren't people going to expect [[1,2], [3,4]][0,1] to work?

Or even [[1,2], [3,4]][*(0,1)] :-)

But seriously, I don't know. memorview is a pretty specialized object, its semantics have more to do with Numpy (which has been a major inspiration and use case for the buffer protocol's design) than standard Python containers. I wouldn't expect a list of lists to support tuple indexing.
msg237928 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-03-12 13:02
Multi-dimensional slicing is explicitly mentioned in PEP-3118, so
I guess the intention was to cover this use case as well.
msg237930 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-03-12 13:06
To be sure, the PEP contains some odd proposals like "Unpacking a long-double will return a decimal object", but the one in this issue seems reasonable.
msg237931 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-12 13:26
May be add support of multi-dimensional sub-views?

>>> m = memoryview(bytearray(range(12)))
>>> m2 = m.cast('B', (3,4))
>>> m2[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NotImplementedError: multi-dimensional sub-views are not implemented

Then we could use m2[0][1].
msg237932 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-03-12 13:35
Sub-views are not as easily implemented. Besides, they create an intermediate object which is immediately thrown away. And tuple indexing is a very common idiom in the Numpy universe.
msg237933 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-03-12 13:43
sub-views and multi-dimensional slicing are not that bad: They're already implemented in _testbuffer (indeed with intermediate objects though).
msg237934 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-03-12 13:44
Well, we could certainly do both :) I don't think we should weigh this issue with separate features, though.
msg237935 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-03-12 13:46
Yes, to be clear I'm +1 on this specific feature -- and separate issues. :)
msg238093 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-03-14 18:25
It turns out that msg237933 was too simplistic: There are some
intricacies of numpy array indexing that *are* very complex:

  http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html


In particular, x[::2, 1, ::2] is not necessarily equal to
x[::2][1][::2].
msg238471 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-03-18 21:26
Do you want to review the patch?
msg238475 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-18 21:52
I left few nitpicks on Rietveld, but in general the patch LGTM.

Oh, needed a documentation.
msg238479 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-03-18 22:53
Updated patch addressing Serhiy's comments.
msg238492 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-19 06:56
It would be good to add examples with multidimensional memoryviews. For example:

>>> a = array.array('l', [-11111111, 22222222, -33333333, 44444444])
>>> m = memoryview(a).cast('B').cast('l', [2, 2])
>>> m[0, 0]
-11111111
>>> m[1, 0]
-33333333
>>> m[-1, -1]
44444444
msg238580 - (view) Author: Roundup Robot (python-dev) Date: 2015-03-19 23:10
New changeset 7d4eb5902f82 by Antoine Pitrou in branch 'default':
Issue #23632: Memoryviews now allow tuple indexing (including for multi-dimensional memoryviews).
https://hg.python.org/cpython/rev/7d4eb5902f82
msg238581 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-03-19 23:11
I didn't add the 2d example as the double cast is quite ugly.  Probably we should allow casting when the format stays the same.
History
Date User Action Args
2015-03-19 23:11:26pitrousetstatus: open -> closed
resolution: fixed
messages: + msg238581

stage: patch review -> resolved
2015-03-19 23:10:32python-devsetnosy: + python-dev
messages: + msg238580
2015-03-19 06:56:50serhiy.storchakasetmessages: + msg238492
2015-03-18 22:53:05pitrousetfiles: + memoryview_tuple_indexing2.patch

messages: + msg238479
2015-03-18 21:52:57serhiy.storchakasetmessages: + msg238475
2015-03-18 21:33:27serhiy.storchakasetstage: needs patch -> patch review
2015-03-18 21:26:00pitrousetmessages: + msg238471
2015-03-14 18:25:39skrahsetmessages: + msg238093
2015-03-12 13:46:50skrahsetmessages: + msg237935
2015-03-12 13:44:10pitrousetmessages: + msg237934
2015-03-12 13:43:22skrahsetmessages: + msg237933
2015-03-12 13:35:56pitrousetmessages: + msg237932
2015-03-12 13:26:23serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg237931
2015-03-12 13:06:52skrahsetmessages: + msg237930
2015-03-12 13:02:51skrahsetmessages: + msg237928
2015-03-12 12:57:38pitrousetmessages: + msg237927
2015-03-12 12:52:36r.david.murraysetnosy: + r.david.murray
messages: + msg237926
2015-03-10 22:43:29pitrousetfiles: + memoryview_tuple_indexing.patch
keywords: + patch
messages: + msg237820
2015-03-10 21:46:41pitroucreate