Author Ellison Marks
Recipients Ellison Marks
Date 2017-04-05.23:28:22
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1491434902.87.0.645672242128.issue30000@psf.upfronthosting.co.za>
In-reply-to
Content
In the zlib module, three of the methods support the wbits parameter, those being zlib.compressobj, zlib.decompress and zlib.decompressobj. zlib.compress does not support the wbits parameter. Looking at the source for these functions, those that support the wbits parameter use the "advanced" version of the zlib functions, deflateInit2 and inflateInit2, whereas zlib.compress uses the "basic" function deflateInit.

The effect of this is that while you can decode from zlib data with non-default wbits values in one call with zlib.decompress, you cannot encode to zlib data with non-default wbits in one call with zlib.compress.  You need to take to extra step of creating a compression object with the appropriate values, then use that to compress the data. eg:

zlib.compress(data) # can't use wbits here

vs.

compressor = zlib.compressobj(wbits=16+zlib.MAX_WBITS)
compressor.compress(data) + compressor.flush()

Some quick benchmarking shows little speed difference between the two implementations:

$ python -m timeit -s 'import zlib' -s 'import random' -s 'import string' -s 's="".join(random.choice(string.printable) for _ in xrange(10000000))' 'zlib.compress(s)'
10 loops, best of 3: 356 msec per loop

$ python -m timeit -s 'import zlib' -s 'import random' -s 'import string' -s 's="".join(random.choice(string.printable) for _ in xrange(10000000))' 'compressor=zlib.compressobj()' 'compressor.compress(s)+compressor.flush()'
10 loops, best of 3: 364 msec per loop

so I can't see any downside of switching zlib.compress to the "advanced" implementation and exposing the extra parameters to python.
History
Date User Action Args
2017-04-05 23:28:22Ellison Markssetrecipients: + Ellison Marks
2017-04-05 23:28:22Ellison Markssetmessageid: <1491434902.87.0.645672242128.issue30000@psf.upfronthosting.co.za>
2017-04-05 23:28:22Ellison Markslinkissue30000 messages
2017-04-05 23:28:22Ellison Markscreate