I'm writing Python code to parse binary (byte oriented) data.

I am (at least somewhat) aware of the performance implications of various approaches to doing the parsing.  With performance in mind, I would like to avoid unnecessary creation/destruction/copying of memory/objects.

An example:

Let's say I am parsing b'0123456789'.
I want to extract and return the substring b'234'.

Now let's say I do this with memoryviews, to avoid unnecessary creation and copying of memory.

m0 = memoryview (b'0123456789')
m1 = m0[2:5]    # m1 == b'234'

Let's say I do this 1000 times.  Each time I use readinto to load the next data into m0.  So I can create m0 only once and reuse it.

But if the relative position of m1 inside m0 changes with each parse, then I need to create a new m1 for each parse.

In the context of the above example, I think it might be nice if I could rebind an existing memoryview to a new object.  For example:

m0 = memoryview (b'0123456789')
m1.bind (m0, 2, 5)    # m1 == b'234'

Is this an idea worth considering?

(Possibly related: Issue 9789, 9757, 3506; PEP 3118)

> In the context of the above example, I think it might be nice if I
> could rebind an existing memoryview to a new object.

It would be nice how so? Can you try to estimate the speed gain?

It would be nice in terms of avoiding malloc()s and free()s.

I could estimate it in terms of memoryview creations per message parse.  I'll be creating 10-20 memoryviews to parse each ~100 byte message.

So... I guess I'd have to build a test to see how long a memoryview creation/free takes.  And then perhaps compare it with variable to variable assignment instead.

If Python pools and recycles unused object by type (the way Lisp recycles cons cells) without free()ing them back to the heap, then there would be minimal speed improvement from my suggestion.  I don't know how CPython works internally, however.

-1 on complicating the code further. It would be possible to pass
an existing memoryview to mbuf_add_view(). That would save the line

   mv = memory_alloc().

But:

  a) You need to check that ndim is correct (shape, strides and
     suboffsets are allocated via the struct hack).

  b) You need to check for existing exports of the memoryview.


  c) ... probably other things that would surface on closer examination.

You could experiment with multiple freelists, one for each ndim.
I'm skeptical however that the gain will be substantial. I've tried
freelists for _decimal and the gain was in the order of 2% (not worth
it).

Yes, I also doubt this would actually bring anything significant, which is why I asked for numbers :-) Python creates many objects in a very intensive fashion, and memoryview are not, by far, the most common object type.

To my surprise, this line is 10% faster with a freelist:

./python -m timeit -s "import array; a = array.array('B', [0]*100); m = memoryview(a)" "m[30:40]"


I think the reason is that PyObject_GC_NewVar() is quite slow.