Author dabeaz
Recipients alex, carljm, coderanger, dabeaz, eric.smith, flox, loewis, mahmoudimus, pitrou
Date 2010-02-17.04:01:13
SpamBayes Score 2.35861e-11
Marked as misclassified No
Message-id <1266379278.52.0.0540866647566.issue7946@psf.upfronthosting.co.za>
In-reply-to
Content
The comment on the CPU-bound workload is valid--it is definitely true that Python 2.6 results will degrade as the workload of each tick is increased.   Maybe a better way to interpreter those results is as a baseline of what kind of I/O performance is possible if there is a quick I/O response time.

However, ignoring that and the comparison between Python 2.6 and 3.2, there is still a serious performance issue with I/O in 3.2.  For example, the dramatic decrease in I/O performance as there are more CPU bound threads competing and the fact that there is a huge performance gain when all but one CPU core is disabled. 

I tried the test using UDP packets and get virtually the exact same behavior described.  For instance, echoing 10MB (sent in 8k UDP packets) takes about 0.6s in Python 2.6 and 12.0s in Python-3.2.   The time shoots up to more than 40s if there are two CPU-bound threads. 

The problem being described really doesn't have anything to do with TCP vs. UDP or any part of the network stack.  It has everything to do with how the operating system buffers I/O requests and how I/O operations such as sends and receives complete immediately without blocking depending on system buffer characteristics (e.g., if there is space in the send buffer, a send will return immediately without blocking).   The fact that the GIL is released when it's not necessary in these cases is really the source of the problem.
History
Date User Action Args
2010-02-17 04:01:18dabeazsetrecipients: + dabeaz, loewis, pitrou, eric.smith, carljm, coderanger, alex, flox, mahmoudimus
2010-02-17 04:01:18dabeazsetmessageid: <1266379278.52.0.0540866647566.issue7946@psf.upfronthosting.co.za>
2010-02-17 04:01:14dabeazlinkissue7946 messages
2010-02-17 04:01:13dabeazcreate