I'd second the proposal of considering the "array.array" type for this, instead of the bytes/bytearray types. Why? Because it is somewhat of a niche case after all, and the bytes type seems too exposed for it. Also, array.array supports more diverse base item types than just plain bytes, which could potentially cover more use cases or at least simplify certain data processing needs.

Alternatively, implement a lightweight SIMD-like memoryview type that operates on arbitrary buffers (which covers bytes, bytearray and array.array). But that's either NumPy then, or something new that would best spend its early days on PyPI.