While tweaking the BufferedWriter implementation it came to me that it
would be useful to have rotate_left and rotate_right methods on
bytearray, so as to rotate the array by a number of bytes without any
wasteful memory allocations and copies.

(or, if memoryview is one day implemented it could be a memoryview
method instead...)

Sadly, this isn't quite as easy as it would seem. The O(1) memory
overhead version of this requires 2n reads and 2n writes, but does both
reads and writes at two memory locations at a time, which may have
nontrivial performance implications.

The simple version that copies out the small part of the shift into a
temporary buffer, doing a memcpy/memmov internally, then copying the
small data back is likely to have much better performance (worst-case
1.5n reads and 1.5n writes.

Offering this ability in the momoryview object would be very
interesting, though I'm not sure that the memoryview object is able to
offer a multi-segment buffer interface where the segments are not the
same length (this could be hacked by having a single pointer per byte,
but at that point we may as well perform a copy).

Hi,

Sadly, this isn't quite as easy as it would seem.

You are right, I was overly optimist when thinking about this.

Offering this ability in the momoryview object would be very
interesting, though I'm not sure that the memoryview object is able to
offer a multi-segment buffer interface where the segments are not the
same length (this could be hacked by having a single pointer per byte,
but at that point we may as well perform a copy).

I'm not sure what you mean, but I think we can just restrict it to the
simple case of a single contiguous buffer.

shift{left,right} could be useful too (and faster).

In order for MemoryView to know what bytes it is pointing to in memory,
it (generally) keeps a pointer with a length. In order to rotate the
data without any copies, you need a pointer and length for each rotation
plus the original. For example, the equivalent to a rotate left of 8
characters using slicing is... x[8:] + x[:8]. That is two segments.
That's a "multi-segment buffer interface". But typical multi-segment
buffer interfaces require each segment to be exactly the same length
(like numpy), which is not the case with rotations.

Le vendredi 08 août 2008 à 21:44 +0000, Josiah Carlson a écrit :

Josiah Carlson <josiahcarlson@users.sourceforge.net> added the comment:

In order for MemoryView to know what bytes it is pointing to in memory,
it (generally) keeps a pointer with a length. In order to rotate the
data without any copies, you need a pointer and length for each rotation
plus the original. For example, the equivalent to a rotate left of 8
characters using slicing is... x[8:] + x[:8].

Hmm, I think it's simpler if the rotate is done in-place rather than
returning a new object. Most uses of memoryviews are going to be with
APIs requiring a single contiguous segment.
(of course for read-only buffers it would raise an error)

Am -1 on this. Rotating byte arrays has very few use cases and the ones
it does have can typically be met by indexing.

Antoine, do you disagree with Raymond or should we close this?
In any case, I believe this would delayed by the moratorium.

Antoine, do you want to pursue, or can we close this?

I think we can close. bpo-17100 would have been more useful actually.

I think you rather need the inplace shift operation. Or even the move the tail of buffer to the start without filling the remaining. I.e. something like

buffer[:size] = buffer[-size:]

but without creating immediate bytes object. Now it may be written as:

buffer[:size] = memoryview(buffer)[-size:]