This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author bjkeen
Recipients bjkeen
Date 2020-04-29.19:01:32
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1588186893.13.0.865332377255.issue40440@roundup.psfhosted.org>
In-reply-to
Content
Currently the array.array object can export a memoryview, but there is no way to construct one from a memoryview without making a copy of the underlying data.  So in that sense array.array only supports one half of the buffer protocol and this is to allow for the other half.

This proposal is to allow the array object to be constructed from an existing exported buffer without copying and reallocating the memory, permitting operations that can modify the underlying buffer's contents but not the allocation.

This is useful when working with many small pieces of one very large underlying buffer that you do not want to copy, when desiring to work with different parts of it with different types, and as part of a way to work with shared memory in multiple processes.

I will shortly have a PR for this, including updates for the documentation and unit tests.

 - Modules/arraymodule.c already must check if the array object has exported a buffer for methods that might resize. If the array was constructed from an imported buffer, the same restrictions apply.  So the object just needs to know whether it is constructed from a Py_Buffer or not and check in the same places it checks for the export count being nonzero. So the code doesn't need to be perturbed that much.

- Only MemoryView objects with contiguous layout, size, and alignment compatible with the data type of the array element are allowed.

- I'm proposing this is only for when it's an actual memoryview object, not just if the object can export buffers. This preserves more of the existing behavior.

- Currently you /can/ initialize an array with a type-compatible memoryview - but it makes a copy, iterating the elements and the types have to match, not just in size. We could maintain exact backward compatibility by adding an extra argument to array.array() or another letter to the format specifier; my current patch doesn't do this though.

-----------------------------------------------------------
Example of current behavior:

>>> import array
>>> x = array.array('b', [1,2,3,4])
>>> y = memoryview(x)
>>> z = array.array('b', y)
>>> z
array('b', [1, 2, 3, 4])
>>> z[0] = 42
>>> x
array('b', [1, 2, 3, 4])
>>> z
array('b', [42, 2, 3, 4])
     # x and z are backed by different memory
>>> x.append(17)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
BufferError: cannot resize an array that is exporting buffers
     # this is because y is still a live object
>>> z.append(17)
     # it is really a copy, x and y are irrelevant to z
>>> z
array('b', [42, 2, 3, 4, 17])

----------------------------------------
Example of new behavior:

>>> import array
>>> x = array.array('b', [1,2,3,4])
>>> x
array('b', [1, 2, 3, 4])
>>> y = memoryview(x)
>>> z = array.array('b', y)
>>> z
array('b', [1, 2, 3, 4])
>>> z[0] = 42
>>> x
array('b', [42, 2, 3, 4])
>>> x.append(4)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
BufferError: cannot resize an array that is exporting buffers
>>> z.append(4)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
BufferError: cannot resize an array constructed from an imported buffer
History
Date User Action Args
2020-04-29 19:01:33bjkeensetrecipients: + bjkeen
2020-04-29 19:01:33bjkeensetmessageid: <1588186893.13.0.865332377255.issue40440@roundup.psfhosted.org>
2020-04-29 19:01:33bjkeenlinkissue40440 messages
2020-04-29 19:01:32bjkeencreate