Author neologix
Recipients neologix, pitrou, sbt, torsten
Date 2013-03-05.17:50:38
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1362505839.05.0.545837614356.issue10886@psf.upfronthosting.co.za>
In-reply-to
Content
For the record, I'm posting thse benchmark numbers here (originally from issue #17025):

"""
with patch:
$ ./python /tmp/multi_queue.py
took 0.7945001125335693 seconds with 1 workers
took 0.7428359985351562 seconds with 2 workers
took 0.7897098064422607 seconds with 3 workers
took 1.1860828399658203 seconds with 4 workers

I tried Richard's suggestion of serializing the data inside put(), but this reduces performance quite notably:
$ ./python /tmp/multi_queue.py
took 1.412883996963501 seconds with 1 workers
took 1.3212130069732666 seconds with 2 workers
took 1.2271699905395508 seconds with 3 workers
took 1.4817359447479248 seconds with 4 workers

Although I didn't analyse it further, I guess one reason could be that if the serializing is done in put(), the feeder thread has nothing to do but keep waiting for data to be available from the buffer, send it, and block until there's more to do: basically, it almost doesn't use its time-slice, and spends its time blocking and doing context switches.
"""

So serializing the data from put() seems to have a significant performance impact (other benchmarks are welcome), that's something to keep in mind.
History
Date User Action Args
2013-03-05 17:50:39neologixsetrecipients: + neologix, pitrou, torsten, sbt
2013-03-05 17:50:39neologixsetmessageid: <1362505839.05.0.545837614356.issue10886@psf.upfronthosting.co.za>
2013-03-05 17:50:39neologixlinkissue10886 messages
2013-03-05 17:50:38neologixcreate