Author vstinner
Recipients alexandre.vassalotti, amaury.forgeotdarc, hagen, pitrou, vstinner
Date 2009-04-05.01:23:07
SpamBayes Score 3.12068e-05
Marked as misclassified No
Message-id <1238894590.15.0.456381103673.issue3873@psf.upfronthosting.co.za>
In-reply-to
Content
alexandre.vassalotti wrote:
> The solution is to add a read buffer to Unpickler (...) 
> would mitigate much of the (quite large) Python function 
> call overhead. (...) cPickle has a performance hack to make it 
> uses cStringIO and PyFile directly (via C function calls). In 
> Python 3, the hack was removed (...)

Yes, unpickler_read() calls Buffered_read() through 
PyObject_Call+PyCFunction_Call which is expensive. And unpickle main 
loop starts with unpickler_read(self, &s, 1): just read *one* byte 
(the "opcode"). If Buffered_read() call is expensive, a solution is to 
avoid calling it (eg. read more bytes and... go backward at the end?).
History
Date User Action Args
2009-04-05 01:23:10vstinnersetrecipients: + vstinner, amaury.forgeotdarc, pitrou, alexandre.vassalotti, hagen
2009-04-05 01:23:10vstinnersetmessageid: <1238894590.15.0.456381103673.issue3873@psf.upfronthosting.co.za>
2009-04-05 01:23:08vstinnerlinkissue3873 messages
2009-04-05 01:23:07vstinnercreate