classification
Title: Locking should be removed from the new buffer protocol
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.0, Python 2.6
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: teoliphant Nosy List: gregory.p.smith, pitrou, scoder, teoliphant
Priority: critical Keywords: patch

Created on 2008-06-06 06:47 by scoder, last changed 2008-08-24 21:35 by teoliphant. This issue is now closed.

Files
File name Uploaded Description Edit
pep-3118-no-locking.patch scoder, 2008-06-06 06:47
buffer-no-locking.patch scoder, 2008-06-06 08:08
Messages (5)
msg67747 - (view) Author: Stefan Behnel (scoder) * Date: 2008-06-06 06:46
Here is a patch against the current PEP 3118 that removes the LOCK flag.
It follows this discussion on the Py3k mailing list:

http://comments.gmane.org/gmane.comp.python.python-3000.devel/13409?set_lines=100000

It has not yet been approved by the PEP owners and requires
implementation, preferably before beta1.
msg67749 - (view) Author: Stefan Behnel (scoder) * Date: 2008-06-06 07:47
As a quick summary of the problems with the current PEP:

1) Many use cases will not require any locking at all, either because
they run single-threaded with a short-read/short-write pattern, or
because they do not write at all.

2) Write locks require exclusive access rights, but there isn't
currently a way to change an existing read lock into a write lock. This
means that to acquire a write lock, all consumers (including the
requester) must first release all read locks before a write lock can be
granted. Therefore, it is not necessary to have such a thing as a read
lock in the first place, as any read request essentially becomes a
read-lock from the POV of a write lock request. And for data integrity
reasons, some kind of write lock must always be applied when writing is
requested, regardless of requesting a lock or not.

3) The requirement in point 2) for releasing all locks before granting a
write lock necessitates short-read/short-write access, in which case
locking is of limited usefulness already.

4) More complex locking scenarios may also require special locking
semantics that are not currently handled by the proposed locking protocol.

The proposal is therefore to

a) remove the locking protocol all-together
b) leave it to application space how read/write locking should be
handled (if required at all).
c) leave it to providers how a request for a writable buffer should be
handled: by just granting it (thus jeopardising data integrity), by
applying a lock internally, or by copying buffers.
msg67754 - (view) Author: Stefan Behnel (scoder) * Date: 2008-06-06 08:08
Here is a patch that removes all occurrences of the locking protocol
from the current py3k branch code base.

There are still some issues in memoryobject.c:

- there was an occurrence of PyBUF_SHADOW that might have to be handled

- memory_getbuf and memory_releasebuf must be reimplemented as it is no
longer allowed to call getbuffer/releasebuffer with a NULL pointer
msg67778 - (view) Author: Travis Oliphant (teoliphant) * (Python committer) Date: 2008-06-06 16:24
Greg Ewing's comment in the thread that read/write locking is orthogonal
to memory-buffer moving is to me the most convincing argument that the
locking portion of the getbuffer function call should be removed and
potentially placed in a separate API at some point.   This will simplify
the protocol a bit.  

I will apply the patch.
msg71440 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-08-19 16:12
It seems to me that the patch has been applied. Can this be closed?
History
Date User Action Args
2008-08-24 21:35:12teoliphantsetstatus: open -> closed
2008-08-19 16:12:31pitrousetnosy: + pitrou
messages: + msg71440
2008-07-07 05:11:29gregory.p.smithsetpriority: critical
assignee: teoliphant
nosy: + gregory.p.smith
2008-06-06 16:24:17teoliphantsetnosy: + teoliphant
messages: + msg67778
2008-06-06 08:08:32scodersetfiles: + buffer-no-locking.patch
messages: + msg67754
2008-06-06 07:47:12scodersetmessages: + msg67749
2008-06-06 06:47:07scodercreate