Author pv
Recipients kermode, loewis, mark.dickinson, ncoghlan, pitrou, pv, rupole, teoliphant
Date 2011-02-14.15:13:21
SpamBayes Score 0.0
Marked as misclassified No
Message-id <1297696402.36.0.412718898483.issue10181@psf.upfronthosting.co.za>
In-reply-to
Content
Nick's plan of action above seems mostly OK to me.

Though, it might be simpler to collapse the PyManagedBuffer object to PyMemoryView, even if this requires memoryviews to serve two purposes.

[Nick:]
> I'm still not comfortable with a convention that relies on *clients*
> of the PEP 3118 API not mucking with the internals of the Py_buffer
> struct.

Some points against: (i) Having to look up keys by memory address from an additional PyDict is more work for the exporter than just passing around some PyMem_Malloc'd information in `internal`. (ii) There is already an "obj" field in the structure that the consumers are supposed to not mess with. (iii) The exporter cannot use the `internal` field for anything if bf_releasebuffer cannot rely on it being intact.

If the recommended consumer API is changed so that Py_buffer mainly sits inside a PyObject, it becomes more clear that Py_buffer is read-only for the consumer (-- which is what I believe the PEP intended, but did not write down).

[Nick:]
> Altering release() to simply decrement the reference count of the 
> managed buffer would defeat the whole point of having that method, so
> it may be necessary to allow early release with outstanding references
> and then include a "still alive" check in the code that allows access
> to the buffer details (similar to the way weak references work).

Early release does not seem possible if the buffer does not come from the original object:

    lst = []
    with memoryview(a) as m:
        b = numpy.array(m)
        lst.append(b)

Now, m.__exit__ cannot release the buffer, since `b` holds a buffer-interface lock to `m`. `b` is 3rd party code, and does not know anything about MemoryViews.

Some alternatives: (i) require that bf_getbuffer always gives a new lock on all exported buffers, if there are multiple, (ii) allow memoryview.__exit__ to fail silently, (iii) drop the context management.

I guess (i) would be a sane choice -- I don't see many use cases for the same object exporting multiple different buffers. It's not needed by Numpy, and I suspect there is no existing 3rd party code that relies on this (because it doesn't work with the current implementation of memoryview :)
History
Date User Action Args
2011-02-14 15:13:22pvsetrecipients: + pv, loewis, teoliphant, mark.dickinson, ncoghlan, rupole, kermode, pitrou
2011-02-14 15:13:22pvsetmessageid: <1297696402.36.0.412718898483.issue10181@psf.upfronthosting.co.za>
2011-02-14 15:13:21pvlinkissue10181 messages
2011-02-14 15:13:21pvcreate