Author jvdias
Recipients jvdias
Date 2009-04-09.06:43:05
SpamBayes Score 5.4992e-11
Marked as misclassified No
Message-id <>
Example Python use of Parallel ForEach : Log Compressor


$ gcc -o -fPIC -shared psempy.c -I/usr/include/python2.6
-L/usr/lib/python2.6 -lpython2.6
$ dd if=/dev/urandom bs=1000000 count=1 of=app1_20090407.01.log
$ python 
Python 2.6 (r26:66714, Oct 16 2008, 00:21:12)
[GCC 4.2.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys, os, re, datetime, psem, psem_example
>>> psem_example.compress_log( "app1", "2009", "04", "07", "01", "bzip",
>>> quit
Use quit() or Ctrl-D (i.e. EOF) to exit

Now, one can time for example 16 runs of the above command with 
16 different input files, and on a 16 CPU machine one would expect
the elapsed time between start and finish to be of the order of
1/16th of the time taken to compress all of the 16 files sequentially.
Date User Action Args
2009-04-09 06:43:11jvdiassetrecipients: + jvdias
2009-04-09 06:43:11jvdiassetmessageid: <>
2009-04-09 06:43:09jvdiaslinkissue5725 messages
2009-04-09 06:43:09jvdiascreate