classification
Title: Explicit buffer release for memoryview objects
Type: enhancement Stage:
Components: Versions: Python 3.2
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Add context manager protocol to memoryviews
View: 9757
Assigned To: Nosy List: gvanrossum, ncoghlan, pitrou
Priority: normal Keywords:

Created on 2010-09-07 12:53 by ncoghlan, last changed 2010-09-07 13:37 by ncoghlan. This issue is now closed.

Messages (4)
msg115753 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-09-07 12:53
memoryview objects currently offer no way to explicitly release the underlying buffer.

This may cause problems for mutable objects that are locked while PEP 3118 buffer references remain unreleased (e.g. in 3.2, io.BytesIO supports getbuffer() for direct access to the underlying memory, but disallows resizing until the associated memoryview goes away).

This isn't too bad in CPython due to explicit refcounting, but may be an issue when using other implementations since the time between release of the last reference and actual garbage collection is indeterminate. For example, the new test_getbuffer in the BytesIOMixin class in the test suite relies on "del buf" promptly releasing the underlying PEP 3118 buffer, which may not be the case on other implementations.

So there are two separate questions here:
1. Whether or not to add an explicit "release()" method to memoryview objects (this would be sufficient to address the problem)
2. Whether or not to add direct context management support to memoryview objects (this isn't really necessary, since a short context manager can do the same thing, but may be a nice convenience)

Nosy list members added based on relevant python-dev discussion where GvR previously nixed part 2 of the idea. At the time, I hadn't really separated the two questions properly in my mind, but I suspect the failing to do something about the first one will prove problematic in the long run (or even the short run, as the test suite issue suggests).
msg115754 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-09-07 12:55
The test suite actually does force a GC collection cycle, so the test may be more portable than I first thought, but the need to do that suggests an explicit buffer release API may be a more appropriate solution.
msg115755 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-09-07 13:10
See the context management patch at #9757.
msg115757 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-09-07 13:28
Thanks for that link. Compared to what either of us managed in the original thread, I think my first post here better articulates why the ability to explicitly release the buffer is important - not because of general memory usage, but because the object owning the buffer (e.g. a BytesIO instance) may behave differently while the buffer reference exists.

It wasn't until I saw the actual getbuffer() patch on python-checkins that this rationale actually clicked for me.
History
Date User Action Args
2010-09-07 13:37:30ncoghlansetsuperseder: Add context manager protocol to memoryviews
2010-09-07 13:37:10ncoghlansetstatus: open -> closed
resolution: duplicate
2010-09-07 13:28:16ncoghlansetmessages: + msg115757
2010-09-07 13:10:12pitrousetmessages: + msg115755
2010-09-07 12:55:21ncoghlansetmessages: + msg115754
2010-09-07 12:53:13ncoghlancreate