Author dabeaz
Recipients DazWorrall, alex, brian.curtin, carljm, coderanger, dabeaz, eric.smith, flox, jhylton, karld, kevinwatters, loewis, mahmoudimus, pitrou, rcohen, rh0dium, tarek
Date 2010-03-15.16:38:47
SpamBayes Score 5.21074e-07
Marked as misclassified No
Message-id <>
Whoa, that's pretty diabolically evil with bufsize=1.  On my machine, doing that just absolutely kills the performance (13 seconds without the spinning thread versus 557 seconds with the thread!).  Or, put another way, the writing performance drops from about 0.5 Mbyte/sec down to 12 Kbytes/sec with the thread.   With my priority GIL, the time is about 29 seconds with the thread (consistent with your experiment using the gilinter patch).
One thing to think about with this example is the proper priority of I/O handling generally.  What if, instead of a file, this example code was writing on a pipe to another process?   For that, you would probably want that I/O thread to be able to blast its data to the receiver as fast as it reasonably can so that it can be done with it and get back to other work. 

In terms of practical relevance, this test again represents a simple situation where computation is overlapped with I/O processing.  Perhaps the program has just computed a big result which is now being streamed somewhere else by a background thread.  In the meantime, the program is now working on computing the next result (the spinning thread).  Think queues, actors, or any number of similar things---there are programs that try to operate like this.
Date User Action Args
2010-03-15 16:38:49dabeazsetrecipients: + dabeaz, loewis, jhylton, pitrou, eric.smith, kevinwatters, tarek, karld, carljm, coderanger, alex, brian.curtin, flox, DazWorrall, rh0dium, rcohen, mahmoudimus
2010-03-15 16:38:49dabeazsetmessageid: <>
2010-03-15 16:38:48dabeazlinkissue7946 messages
2010-03-15 16:38:47dabeazcreate