Author kristjan.jonsson
Recipients kristjan.jonsson
Date 2009-01-08.10:43:48
SpamBayes Score 0.00500315
Marked as misclassified No
Message-id <1231411436.11.0.236061918484.issue4879@psf.upfronthosting.co.za>
In-reply-to
Content
This is related to issue 4336.  While that issue finally revolved around 
fixing the Nagle problem for xmlrpc and other http clients, here I 
propose to fix another performance bottleneck with xmlrpc, reading the 
HTTP Headers.
HTTPResponse creates a socket.fileobject() with zero buffering which 
means that the readline() operations used to read the headers become 
very inefficient since individual socket.recv() calls are used for each 
character.
The reason for this is to support users who subsequently do 
socket.recv() on the underlying socket, and so we don't want to read too 
much of the headers. Note that there is no example of this in the 
standard library.
In the provided patch, I have removed some vestigial code from 
xmrlpclib.py which attempted to use recv() even though it never did 
(because the connection has been closed when HTTPConnection returns the 
response).  Even so, Guido has expressed his concern that we need to 
keep the support for this recv() behaviour in place.
Therefore, I have added a new optional argument, nobuffer=True, which 
users that promise not to call recv() can set to false.  This will then 
cause the generated fileobject from HTTPResponse to have default 
buffering, greatly increasing the performance of the reading of the 
headers.
History
Date User Action Args
2009-01-08 10:43:56kristjan.jonssonsetrecipients: + kristjan.jonsson
2009-01-08 10:43:56kristjan.jonssonsetmessageid: <1231411436.11.0.236061918484.issue4879@psf.upfronthosting.co.za>
2009-01-08 10:43:55kristjan.jonssonlinkissue4879 messages
2009-01-08 10:43:53kristjan.jonssoncreate