Author dabeaz
Recipients DazWorrall, aconrad, alex, andrix, brian.curtin, carljm, coderanger, cool-RR, dabeaz, djc, durin42, eric.araujo, eric.smith, flox, gregory.p.smith, jcea, jhylton, karld, kevinwatters, konryd, larry, loewis, mahmoudimus, movement, neologix, nirai, pitrou, rcohen, rh0dium, salgado, tarek, thouis, ysj.ray
Date 2010-04-26.13:31:14
SpamBayes Score 3.37642e-05
Marked as misclassified No
Message-id <1272288676.57.0.140012794498.issue7946@psf.upfronthosting.co.za>
In-reply-to
Content
I've also attached a new file schedtest.py that illustrates a subtle difference between having the GIL monitor thread and not having the monitor.

Without the monitor, every thread is responsible for its own scheduling.  If you have a lot of threads running, you may have a lot of threads all performing a timed wait and then waking up only to find that the GIL is locked and that they have to go back to waiting.  One side effect is that certain threads have a tendency to starve.

For example, if you run the schedtest.py with the original GIL, you get a trace where three CPU-bound threads run like this:

Thread-3 16632
Thread-2 16517
Thread-1 31669
Thread-2 16610
Thread-1 16256
Thread-2 16445
Thread-1 16643
Thread-2 16331
Thread-1 16494
Thread-3 16399
Thread-1 17090
Thread-1 20860
Thread-3 16306
Thread-1 19684
Thread-3 16258
Thread-1 16669
Thread-3 16515
Thread-1 16381
Thread-3 16600
Thread-1 16477
Thread-3 16507
Thread-1 16740
Thread-3 16626
Thread-1 16564
Thread-3 15954
Thread-2 16727
...

You will observe that Threads 1 and 2 alternate, but Thread 3 starves.  Then at some point, Threads 1 and 3 alternate, but Thread 2 starves. 

By having a separate GIL monitor, threads are no longer responsible for making scheduling decisions concerning timeouts.  Instead, the monitor is what times out and yanks threads off the GIL.  If you run the same test with the GIL monitor, you get scheduling like this:

Thread-1 33278
Thread-2 32278
Thread-3 31981
Thread-1 33760
Thread-2 32385
Thread-3 32019
Thread-1 32700
Thread-2 32085
Thread-3 32248
Thread-1 31630
Thread-2 32200
Thread-3 32054
Thread-1 32721
Thread-2 32659
Thread-3 34150

Threads nicely cycle round-robin.  There also appears to be about half as much thread switching (for reasons I don't quite understand).
History
Date User Action Args
2010-04-26 13:31:16dabeazsetrecipients: + dabeaz, loewis, jhylton, gregory.p.smith, jcea, pitrou, movement, larry, eric.smith, kevinwatters, tarek, djc, karld, carljm, coderanger, durin42, eric.araujo, nirai, alex, andrix, konryd, brian.curtin, flox, DazWorrall, salgado, cool-RR, rh0dium, rcohen, mahmoudimus, aconrad, ysj.ray, neologix, thouis
2010-04-26 13:31:16dabeazsetmessageid: <1272288676.57.0.140012794498.issue7946@psf.upfronthosting.co.za>
2010-04-26 13:31:15dabeazlinkissue7946 messages
2010-04-26 13:31:14dabeazcreate