Author florianfesti
Recipients BreamoreBoy, florianfesti, georg.brandl, karld, lucas_malor, nils, pitrou, rharris
Date 2010-09-22.10:09:25
SpamBayes Score 7.14145e-10
Marked as misclassified No
Message-id <1285150170.1.0.62688743466.issue1675951@psf.upfronthosting.co.za>
In-reply-to
Content
Stupid me! I ran the tests against my systems gzip version (Py 3.1). The performance issue is basically fixed by rev 77289. Performance is even a bit better that my original patch by may be 10-20%. The only test case where it performs worse is 

Random 10485760 byte block test
Original gzip Write:   20.452 s Read:    2.931 s
New gzip      Write:   20.518 s Read:    1.247 s

Don't know if it is worth bothering. May be increasing the maximum chunk size improves this - but I didn't try that out yet.

WRT to seeking:

I now have two patches that eliminate the need for seek() on normal operation (rewind obviously still needs seek()). Both are based on the PaddedFile class. The first patch just creates a PaddedFile object while switching from an old to a new member while the second just wraps the fileobj all the time. Performance test show that wrapping is cheap. The first patch is a bit ugly while the second requires a implementation of seek() and may create problems if new methods of the fileobj are used that may interfere with the PaddedFile's internals.

So I leave the choice which one is preferred to the module owner.

The patch creates another problem with is not yet fixed: The implementation of .seekable() is becoming wrong. As one can now use non seekable files the implementation should check if the file object used for reading is really seekable. As this is my first PY3k work I'd prefer if this can be solved by someone else (But that should be pretty easy).
History
Date User Action Args
2010-09-22 10:09:30florianfestisetrecipients: + florianfesti, georg.brandl, pitrou, lucas_malor, rharris, karld, nils, BreamoreBoy
2010-09-22 10:09:30florianfestisetmessageid: <1285150170.1.0.62688743466.issue1675951@psf.upfronthosting.co.za>
2010-09-22 10:09:29florianfestilinkissue1675951 messages
2010-09-22 10:09:27florianfesticreate