Author ebfe
Recipients ebfe
Date 2008-12-24.16:29:28
SpamBayes Score 8.57208e-05
Marked as misclassified No
Message-id <1230136171.32.0.121363130694.issue4738@psf.upfronthosting.co.za>
In-reply-to
Content
My application needs to pack and unpack workunits all day long and does
this using multiple threading.Threads. I've noticed that the zlib module
seems to use only one thread at a time when using [de]compressobj(). As
the comment in the sourcefile zlibmodule.c already says the module uses
a global lock to protect different threads from accessing the object.
While the c-functions release the GIL while waiting for the global lock,
only one thread at a time can use zlib.
My app ends up using only one CPU to compress/decompress it's workunits...

The patch (svn diff to ) attached here fixes this problem by extending
the compressobj-structure by an additional member to create
object-specific locks and removes the global lock. The lock protects
each compressobj individually and allows multiple python threads to use
zlib in parallel, utilizing all available CPUs.
History
Date User Action Args
2008-12-24 16:29:31ebfesetrecipients: + ebfe
2008-12-24 16:29:31ebfesetmessageid: <1230136171.32.0.121363130694.issue4738@psf.upfronthosting.co.za>
2008-12-24 16:29:30ebfelinkissue4738 messages
2008-12-24 16:29:29ebfecreate