classification
Title: Document better what happens on releasing an unacquired lock
Type: Stage: resolved
Components: Documentation Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Jim.Jewett, asvetlov, docs@python, georg.brandl, petri.lehtinen, pitrou, python-dev, r.david.murray, sandro.tosi, vinay.sajip
Priority: normal Keywords:

Created on 2012-04-05 07:06 by georg.brandl, last changed 2013-10-13 08:24 by georg.brandl. This issue is now closed.

Messages (22)
msg157544 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-04-05 07:06
From docs@python.org:

"""
I recently ran into a situation where I could not be certain that a lock 
was currently in the acquired state. I checked the documentation to 
determine what would happen if I attempted to release a lock that was 
already released, and saw an ominous warning of "Do not call this method 
when the lock is unlocked."

Needing to know what would happen, I cautiously tested it out. I half 
expected my computer to explode as I released a lock for the second 
time, but was pleased to see it raise a 'thread.error' exception which 
could be caught and handled.

I generally expect the documentation to tell me what will happen if I do 
something invalid. In this case the documentation should indicate that a 
thread.error will be raised if you release an unlocked lock.
"""


I agree: if we know that a ThreadError will always be raised in this instance, we should document it as such.
msg157547 - (view) Author: Sandro Tosi (sandro.tosi) * (Python committer) Date: 2012-04-05 07:37
On Thu, Apr 5, 2012 at 09:06, Georg Brandl <report@bugs.python.org> wrote:
> I agree: if we know that a ThreadError will always be raised in this instance, we should document it as such.

I've already prepared a small patch for that (every supported release
has a different exception raised..) I'll be fixing it this evening at
home.
msg157548 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-04-05 07:48
What different exceptions are they? Note that thread.error == _thread.error == threading.ThreadError.  The docs should always use the last one (ThreadError).
msg157549 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-04-05 07:53
Ah, and I missed that apparently on 3.3, _thread.Error is aliased to RuntimeError.  In that case you should use RuntimeError of course :)
msg157623 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-04-05 21:01
New changeset efeca6ff2751 by Sandro Tosi in branch '2.7':
Issue #14502: release() and unlocked lock generates a ThreadError
http://hg.python.org/cpython/rev/efeca6ff2751

New changeset acea9d95a6d8 by Sandro Tosi in branch '3.2':
Issue #14502: release() and unlocked lock generates a ThreadError
http://hg.python.org/cpython/rev/acea9d95a6d8

New changeset c10a0f93544e by Sandro Tosi in branch 'default':
Issue #14502: merge with 3.2
http://hg.python.org/cpython/rev/c10a0f93544e
msg157625 - (view) Author: Jim Jewett (Jim.Jewett) (Python triager) Date: 2012-04-05 21:35
At least put the information inside some disclaimers about "normally"; even the stdlib has some fake locks that let you release a lock someone else holds.  (I think I found them in in workarounds for threading not being available, such as the dummy_* modules, but still, it is possible.)
msg157626 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-04-05 21:38
> At least put the information inside some disclaimers about "normally";
> even the stdlib has some fake locks that let you release a lock
> someone else holds.

Not sure what you're talking about. The doc patch is about unacquired
locks, not locks that someone else (another thread) holds.

Indeed the standard Lock object (but not the RLock) does allow releasing
from another thread. It's a feature (which makes it serve as a binary
semaphore).
msg157643 - (view) Author: Jim Jewett (Jim.Jewett) (Python triager) Date: 2012-04-06 04:59
On Thu, Apr 5, 2012 at 5:38 PM, Antoine Pitrou <report@bugs.python.org> wrote:
> Antoine Pitrou <pitrou@free.fr> added the comment:

> Not sure what you're talking about. The doc patch is about unacquired
> locks, not locks that someone else (another thread) holds.

Isn't one common reason for not being able to acquire a lock that
someone else was already holding it?
msg157652 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-04-06 09:15
New changeset 068a614e9d97 by Sandro Tosi in branch 'default':
Issue #14502: it's RuntimeError on 3.3
http://hg.python.org/cpython/rev/068a614e9d97
msg157654 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-04-06 09:57
> On Thu, Apr 5, 2012 at 5:38 PM, Antoine Pitrou <report@bugs.python.org> wrote:
> > Antoine Pitrou <pitrou@free.fr> added the comment:
> 
> > Not sure what you're talking about. The doc patch is about unacquired
> > locks, not locks that someone else (another thread) holds.
> 
> Isn't one common reason for not being able to acquire a lock that
> someone else was already holding it?

We're talking about *releasing* an (un)acquired lock, not acquiring it
again...
msg157678 - (view) Author: Jim Jewett (Jim.Jewett) (Python triager) Date: 2012-04-06 19:37
On Fri, Apr 6, 2012 at 5:57 AM, Antoine Pitrou <report@bugs.python.org> wrote:
> Antoine Pitrou <pitrou@free.fr> added the comment:

>> > Not sure what you're talking about. The doc patch is about unacquired
>> > locks, not locks that someone else (another thread) holds.

>> Isn't one common reason for not being able to acquire a lock that
>> someone else was already holding it?

> We're talking about *releasing* an (un)acquired lock, not acquiring it
> again...

Right, but I thought the original motivation was concern over a race
condition in the lock acquisition.

    lock.acquire()
    try:                    # What if something happens here, during
try setup?  Leak?
        foo()
    finally:
        lock.release()

vs

    try:
        lock.acquire()
        foo()
    finally:
        lock.release()               # But what if the acquire failed?

-jJ
msg157680 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-04-06 19:43
It doesn't matter *how* you get to the situation where you are releasing a lock that hasn't been acquired, the point is to document what actually happens when you do the release.  And just yesterday I needed to know this, since I have a lock that may or may not be currently held when I release it, and now I know I can just catch RuntimeError in that case.
msg157693 - (view) Author: Jim Jewett (Jim.Jewett) (Python triager) Date: 2012-04-06 21:37
> I have a lock that may or may not be currently held when I release it, 
> and now I know I can just catch RuntimeError in that case.

Only if you're willing to make assumptions about the threading model and the source of locks.  And I fear the current change overpromises.

For example, the LockType from _dummy_thread raises an error not based on RuntimeError, and has comments suggesting it might stop raising entirely.  I believe I have seen other Lock-emulation code which also does not raise an error, though the closest I can come to finding it right now is logging_releaseLock() when the import of either _thread or threading failed.

Starting with http://hg.python.org/cpython/file/acea9d95a6d8/Doc/library/threading.rst

I would prefer to change to following two sentences:

    If an attempt is made to release an unlocked lock, a :exc:`RuntimeError` will be raised.
...
    When invoked on an unlocked lock, a :exc:`ThreadError` is raised.

in any of the following ways:

(a) Change "will be"/"is" --> "may be", so it isn't promised:

    If an attempt is made to release an unlocked lock, a :exc:`RuntimeError` may be raised.
...
    When invoked on an unlocked lock, a :exc:`ThreadError` may be raised.

(b) Clarify that it is implementation-specific

    If an attempt is made to release an unlocked _thread.lock, a :exc:`RuntimeError` will be raised.
...
    When invoked on an unlocked _thread.lock, a :exc:`ThreadError` is  raised.

(and add to the caveats)
    Locks provided by other modules may have slightly different behavior, particularly when an an operation fails.  For example, unlocking without first acquiring may raise a different error, or may not raise at all.

(c) Clarify that alternatives are buggy (and fix those in the stdlib)
    If an attempt is made to release an unlocked lock, a :exc:`RuntimeError` will be raised.
...
    When invoked on an unlocked lock, a :exc:`ThreadError` is be raised.


(and add to the caveats)
    Historically, many Locks have followed a slightly different contract, particularly when an an operation fails.  For example, unlocking without first acquiring might raise a different error, or might not even raise at all.
msg157715 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-04-07 02:32
I, on the other hand, would prefer if it were made part of the API contract that an error is raised, and to fix any stdlib implementations *of that API* that don't conform to that.  (That is, locks from other modules may well not follow that API, and their documentation should cover their API.)
msg157827 - (view) Author: Jim Jewett (Jim.Jewett) (Python triager) Date: 2012-04-08 23:59
On Fri, Apr 6, 2012 at 10:32 PM, R. David Murray <report@bugs.python.org> wrote:

> R. David Murray <rdmurray@bitdance.com> added the comment:

> I, on the other hand, would prefer if it were made part of the API contract that an
> error is raised, and to fix any stdlib implementations *of that API* that don't conform
> to that.  (That is, locks from other modules may well not follow that API, and their
> documentation should cover their API.)

Do you consider it reasonable that all stdlib Locks follow that API,
and change to raise either RuntimeError or a subclass?

I don't feel comfortable declaring that (not even only for future
feature releases), but if you do, or Guido does, or ... etc ... I'll
submit patches for at least dummy_threading and logging.

-jJ
msg157830 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-04-09 01:58
I think dummy_threading should be fixed (but only in 3.3, just in case it causes any backward compatibility issues with someone's code).

Logging I'd leave to Vinay to decide about.

I'm assuming that if any of the others devs nosy on this issue disagree with me that they will speak up :)
msg157877 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2012-04-09 18:57
Re. logging, logging._acquireLock and logging._releaseLock are not part of the public API and are undocumented at present. The case when _releaseLock does not raise an error is when threading couldn't be imported, so the _lock variable is None. I don't see the need for adding any documentation for this.

Logging doesn't use dummy_thread: if threading isn't available, all lock acquisition and release operations become no-ops.
msg157883 - (view) Author: Jim Jewett (Jim.Jewett) (Python triager) Date: 2012-04-09 19:26
Vinay,

The current question is what contract locks should follow, and whether
all locks should follow it.  Would it be acceptable for
logging._releaseLock to raise a RuntimeError if the lock hadn't
previously been acquired?  In other words, would it be acceptable to
replace the current None with a counter (and to note in comments that
it should be safe from race conditions because it is only used when
threading isn't available).

-jJ
msg157884 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-04-09 19:31
> The current question is what contract locks should follow, and whether
> all locks should follow it.  Would it be acceptable for
> logging._releaseLock to raise a RuntimeError if the lock hadn't
> previously been acquired?

I don't see the point of this discussion. We are talking about
threading.Lock (and, possibly, multiprocessing.Lock), not every lock API
under the sun. Especially when it's a private API...
msg157901 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-04-09 20:58
Agreed. Jim, I think you're trying to get consistency where none is required.
msg162417 - (view) Author: Petri Lehtinen (petri.lehtinen) * (Python committer) Date: 2012-06-06 18:00
The docs of 2.7 and 3.2 still first say that RuntimeError is raised, and then that a ThreadError is raised:

    ...
    If an attempt is made to release an unlocked lock, a RuntimeError
    will be raised.

    ...

Lock.release()
    ...
    When invoked on an unlocked lock, a ThreadError is raised.


In 2.7 and 3.2, ThreadError is not a RuntimeError, so this is wrong.
msg199672 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2013-10-13 08:24
This is now fixed for 2.7 (see #15829); no fix needed for 3.3+.
History
Date User Action Args
2013-10-13 08:24:35georg.brandlsetstatus: open -> closed
resolution: fixed
messages: + msg199672
2012-08-31 13:20:16asvetlovsetnosy: + asvetlov
2012-06-06 18:00:51petri.lehtinensetstatus: closed -> open

nosy: + petri.lehtinen
messages: + msg162417

resolution: fixed -> (no value)
2012-06-06 13:14:50r.david.murraylinkissue15017 superseder
2012-04-09 20:58:18georg.brandlsetmessages: + msg157901
2012-04-09 19:31:03pitrousetmessages: + msg157884
2012-04-09 19:26:26Jim.Jewettsetmessages: + msg157883
2012-04-09 18:57:48vinay.sajipsetmessages: + msg157877
2012-04-09 01:58:18r.david.murraysetnosy: + vinay.sajip
messages: + msg157830
2012-04-08 23:59:03Jim.Jewettsetmessages: + msg157827
2012-04-07 02:32:16r.david.murraysetmessages: + msg157715
2012-04-06 21:37:34Jim.Jewettsetmessages: + msg157693
2012-04-06 19:43:47r.david.murraysetnosy: + r.david.murray
messages: + msg157680
2012-04-06 19:37:14Jim.Jewettsetmessages: + msg157678
2012-04-06 09:57:33pitrousetmessages: + msg157654
2012-04-06 09:15:49python-devsetmessages: + msg157652
2012-04-06 04:59:33Jim.Jewettsetmessages: + msg157643
2012-04-05 21:38:17pitrousetmessages: + msg157626
2012-04-05 21:35:14Jim.Jewettsetnosy: + Jim.Jewett
messages: + msg157625
2012-04-05 21:05:47sandro.tosisetstatus: open -> closed
resolution: fixed
stage: resolved
2012-04-05 21:01:58python-devsetnosy: + python-dev
messages: + msg157623
2012-04-05 07:53:16georg.brandlsetmessages: + msg157549
2012-04-05 07:48:22georg.brandlsetmessages: + msg157548
2012-04-05 07:37:36sandro.tosisetnosy: + sandro.tosi
messages: + msg157547
2012-04-05 07:06:56georg.brandlcreate