Author pv
Recipients kermode, loewis, mark.dickinson, ncoghlan, pitrou, pv, rupole, teoliphant
Date 2011-02-13.03:47:31
SpamBayes Score 4.69901e-10
Marked as misclassified No
Message-id <1297568852.54.0.806337018426.issue10181@psf.upfronthosting.co.za>
In-reply-to
Content
I spent today some time to rewrite `memoryobject.c`, and cleaning up the Py_buffer handling in it. (I wrote also the Numpy PEP 3118 implementation, so this was straightforward to do.)

The end result is here: https://bitbucket.org/pv/cpython-stuff/changesets   (bookmark bug/memoryview)

Some points of note:

- Clarification of ownership conventions for Py_buffer in docs:

  http://bitbucket.org/pv/cpython-stuff/changeset/805971191369

  The new implementation is written to respect the invariants
  mentioned there.

- Rewritten memoryobject:

  http://bitbucket.org/pv/cpython-stuff/src/817edc63ce4d/Objects/memoryobject.c

I didn't yet heavily test this (eg. against Numpy), but at least the Python test suite passes. There are also some minor corners that need to be polished. Also some new tests would be needed, as I slightly extended the slicing capabilities of memoryview.

Only one nasty point turned up: the existence of PyMemoryView_FromBuffer. This function breaks the ownership rules: the pointers in Py_buffer structure can point inside the structure --- and the structure can reside e.g. on the stack. Deep copying Py_buffer cannot in general be done, because this can mess up whatever bf_releasebuffer tries to do. The workaround here was to do a deep copy *only* for those pointers that point inside the struct --- although this violates the spirit of the ownership model, it should be essentially always safe to do, and I believe it is the correct solution (at least if PyMemoryView_FromBuffer is to be retained).

    ***

Comments are welcome.
History
Date User Action Args
2011-02-13 03:47:32pvsetrecipients: + pv, loewis, teoliphant, mark.dickinson, ncoghlan, rupole, kermode, pitrou
2011-02-13 03:47:32pvsetmessageid: <1297568852.54.0.806337018426.issue10181@psf.upfronthosting.co.za>
2011-02-13 03:47:32pvlinkissue10181 messages
2011-02-13 03:47:31pvcreate