Today while working on an attempt to improve the GIL (either by modifying it or removing it) I noticed that in destroy_gil function first we unlock the mutex and then we set the gil->locked to -1 using _Py_atomic_store_explicit. Therefore the cycle is, "Unlock -> Atomic_Set_Value" which closely evaluates to, "Unlock -> Lock -> Set_Value -> Unlock"


I tweaked around a little and when I changed the cycle to, "Atomic_Set_Value -> Unlock" I noticed by running David Beazley's famous benchmarks that there's an increase in speed by 0.22 or something like that. I don't know if it's because of CPU instability or because of the tweak but it looks like an improvement. Should this patch be applied to cpython?

Instability is more likely. But I'll run the benchmark once more and see what is happening.

Does Python's unit test suite pass?

Forgot to do that. I'll report back with the test suite result.

"there's an increase in speed by 0.22 or something like that"

0.22 what? Seconds? Percent? A factor of 0.22?

What are you measuring, and how are you measuring it?

I took the measurement like this. If it ran in 1.987777 secs then the new patch would theoretically measure 1.967777 (original - 0.22). The patch isn't very useful but it can sometimes be faster than the present code especially in debug mode.

The information is not very helpful to understand what you are doing. Please provide your system specs (arch, platform, CPU spec, memory), the commands you used to compile Python and the commands you used to get the numbers

Test suite passed and also I'm attaching the benchmark I used to measure. (This is the benchmark used by David Beazley in one of his famous GIL blog posts and talks.)

"If it ran in 1.987777 secs then the new patch would theoretically measure 1.967777 (original - 0.22)"

That difference is 0.02 seconds, or about 1%, correct?

I'm just trying to understand what we're measuring here.

> "That difference is 0.02 seconds, or about 1%, correct?"

I mistook 1.987777 as 1.88 in my calculation. Sorry for that.

> The information is not very helpful to understand what you are doing. Please provide your system specs (arch, platform, CPU spec, memory), the commands you used to compile Python and the commands you used to get the numbers

My System is x86_64 Windows with 4 cores and 8 gigs of memory. I also tried it with my WSL1.

The benchmark code has multiple issues. For example it uses a bad clock sources, time.time(). Could you please rewrite the benchmark to use https://pypi.org/project/pyperf/ ?

Sure

I attached the pyperf benchmark also.

> I attached the pyperf benchmark also.

Can you please provide pyperf_benchmark.py results?

pyperf results -

Mean +- std dev: 455 ms +- 13 ms

Looks a little bit faster than current 3.11, 3.10 and 3.9 versions.

> Mean +- std dev: 455 ms +- 13 ms

A single benchmark is useless without a reference. Please run the benchmark on unpatched python, then run it again on a patched Python, and compare results. You can store benchmark results using -o option. Example:

# Unpatched python
python pyperf_benchmark.py -o ref.json
# Patched python
python pyperf_benchmark.py -o new_gil.json
# Compare
python -m pyperf compare_to ref.json new_gil.json

I suspected right. This was just CPU instability. In fact the patch is 1x-2x slower as reported by pyperf. Therefore I'm closing this issue. 

Though I'm working on another "yet to be failed attempt" to remove or rewrite the GIL to improve performance. Of course, any help would be appreciated there.

Repo - https://github.com/shreyanavigyan/pygilremove