classification
Title: memoryview.__len__ should raise an exception for 0d buffers
Type: behavior Stage: patch review
Components: Interpreter Core Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: skrah Nosy List: Eric Wieser, serhiy.storchaka, skrah
Priority: normal Keywords: patch

Created on 2020-02-11 15:42 by Eric Wieser, last changed 2020-09-07 17:25 by skrah.

Pull Requests
URL Status Linked Edit
PR 18463 open Eric.Wieser, 2020-02-11 15:51
Messages (7)
msg361821 - (view) Author: Eric Wieser (Eric Wieser) Date: 2020-02-11 15:42
Right now, the behavior is:

>>> import numpy as np
>>> arr_0d = np.array(42)
>>> mem_0d = memoryview(arr_0d)
>>> len(mem_0d)
1
>>> mem_0d[0]
TypeError: invalid indexing of 0-dim memory

It seems bizarre to have this object pretend to be a sequence when you ask for its length, yet not behave like one when you actually try to use this length. I'd suggest cpython should behave like numpy here, and fail:

>>> len(arr_0d)
TypeError: len() of unsized object


Perhaps `TypeError: cannot get length of 0-dim memory` would be more appropriate as a message.

---

Wasn't sure how to classify this, feel free to reclassify
msg361840 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-02-12 00:00
The change looks reasonable, but unfortunately this is a long-standing
behavior that originates from before the Python-3.3 memoryview rewrite.

It is also present in 2.7 (the previous implementation) and documented
in 3.3:

https://docs.python.org/3/library/stdtypes.html#memoryview


I think I documented it because it looked strange back then, too.
So I've to think a bit what will be affected and how this behavior
came into place.
msg363159 - (view) Author: Eric Wieser (Eric Wieser) Date: 2020-03-02 11:32
Thanks for pointing out the docs reference, I updated the patch to reword that section.

There's a sentence right before the one you draw attention to which to me reads as another argument to change this:

> ``len(view)`` is equal to the length of :class:`~memoryview.tolist`

On Python 2.7, this gives

>>> len(view_0d)
1
>>> len(view_0d.tolist())
NotImplementedError: tolist() only supports one-dimensional objects

On Python 3.8 before my patch, this gives:

>>> len(view_0d)
1
>>> len(view_0d.tolist())
TypeError: object of type 'int' has no len()

On Python 3.8, with my patch, this gives:

>>> len(view_0d)
TypeError: 0-dim memory has no length
>>> len(view_0d.tolist())
TypeError: object of type 'int' has no len()

As I read it, only with my patch is this sentence satisfied by the implementation.
msg376512 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-09-07 16:25
But iterating it raises different type of exception, wish obscure message.

>>> import ctypes
>>> mem_0d = memoryview(ctypes.c_uint8(42))
>>> list(mem_0d)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NotImplementedError: memoryview: unsupported format <B

Should not '<B' and be equivalent to '>B' and just 'B'? Would not adding support of formats with specified endianess make 0-dimensional memoryview objects iterable?
msg376518 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-09-07 17:17
memoryview can store anything, but only access and display
the native format. The reason is that conversions in the manner
of the struct module are very expensive.

You can construct all sorts of formats that memoryview cannot
handle:

>>> x = array(1, "c")
>>> x
array(b'1', dtype='|S1')
>>> y = memoryview(x)
>>> 
>>> list(y)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NotImplementedError: memoryview: unsupported format 1s


I don't think that the message is obscure, since all these
facts are documented.
msg376519 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-09-07 17:22
It is surprising the endianess of byte matters.
msg376520 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-09-07 17:25
list() has to *interpret* the endianness to display it. The endianness
of the *native* format does not matter.  The endianness of an explicit
format matters, since list has to be able to convert non-native formats.

I suggest to look at XND, which is a memoryview on steroids and
has a lot of those features.
History
Date User Action Args
2020-09-07 17:25:22skrahsetmessages: + msg376520
2020-09-07 17:22:29serhiy.storchakasetmessages: + msg376519
2020-09-07 17:17:49skrahsetassignee: skrah
messages: + msg376518
2020-09-07 16:25:54serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg376512
2020-03-02 11:32:39Eric Wiesersetmessages: + msg363159
2020-02-12 00:00:50skrahsetmessages: + msg361840
2020-02-11 15:51:35Eric.Wiesersetkeywords: + patch
stage: patch review
pull_requests: + pull_request17836
2020-02-11 15:42:52Eric Wiesercreate