Author jeff.allen
Recipients docs@python, graingert, jeff.allen, pablogsal, pitrou, serhiy.storchaka, steven.daprano
Date 2021-10-12.19:34:36
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1634067276.62.0.990719087256.issue45435@roundup.psfhosted.org>
In-reply-to
Content
Thomas wrote:
> it's as part of this discussion in https://mail.python.org/archives/list/python-dev@python.org/thread/ABR2L6BENNA6UPSPKV474HCS4LWT26GY/#IAOCDDCJ653NBED3G2J2YBWD7HHPFHT6 and others in #python-dev 

That's where I noticed it, but it seemed the wrong place to explore this way.

Steven is right, I'm over-stating the case. And although valid that this is CPython specific, it's well sign-posted and I'm just being thin-skinned.

Serhiy writes:
> sort() is atomic, even if GIL is released during executing custom __lt__. It is guaranteed that no operations on the list in other threads can affect the result of sort().

The strategy noted here: https://github.com/python/cpython/blob/2d21612f0dd84bf6d0ce35bcfcc9f0e1a41c202d/Objects/listobject.c#L2261-L2265
does guarantee that, which I hadn't noticed. What if during the release of the GIL, another thread appends to L? In my simple experiment I get a ValueError and the modifications are lost. I think that is not thread-safe.

Serhiy also writes:

> I do not understand what non-atomic you see in x = L[i]. The value of x is determined by values of L and i at the start of the operation. GIL is not released during indexing L, and if it is released between indexing and assignment, it does not affect the result.

and Steven:

> Does that matter though? I think that's a distinction that makes no 
difference.

> We know that another thread could change the L or the i before the 
assignment, if they are global. But once the L[i] lookup has occurred, 
it doesn't matter if they change. It's not going to affect what value 
gets bound to the x.

Fair enough. Atomicity is a bit slippery, I find. It depends where the critical region starts. Thinking again, it's not the assignment that's the issue ...

L is pushed
i is pushed
__getitem__ is called
x is popped

It is possible, if i and L are accessible to another thread and change after L is pushed, that x is given a value composed from an i and an L that never existed concurrently in the view of the other thread. Straining at gnats here, but atomicity is a strong claim.

And on the point about re-ordering and CPUs, I can't imagine re-ordering that effectively changes the order of byte codes. But do CPython threads run in separate CPUs, or is that only when we have multiple interpreters? If so, and L were in a hot memory location (either the variable or its content), this could be inconsistent between threads. Sorry, I don't know the memory coherence CPython has: I know I couldn't rely on it in Java.

I'm just arguing that the section gives advice that is *nearly* always right, which is a horrible thing to debug. I'll stop stirring.
History
Date User Action Args
2021-10-12 19:34:36jeff.allensetrecipients: + jeff.allen, pitrou, steven.daprano, docs@python, serhiy.storchaka, graingert, pablogsal
2021-10-12 19:34:36jeff.allensetmessageid: <1634067276.62.0.990719087256.issue45435@roundup.psfhosted.org>
2021-10-12 19:34:36jeff.allenlinkissue45435 messages
2021-10-12 19:34:36jeff.allencreate