Author jvdias
Recipients jvdias
Date 2009-04-09.06:43:05
SpamBayes Score 5.4992e-11
Marked as misclassified No
Message-id <1239259391.17.0.587935519406.issue5725@psf.upfronthosting.co.za>
In-reply-to
Content
Example Python use of psem.so Parallel ForEach : Log Compressor

Usage:

$ gcc -o psem.so -fPIC -shared psempy.c -I/usr/include/python2.6
-L/usr/lib/python2.6 -lpython2.6
$ dd if=/dev/urandom bs=1000000 count=1 of=app1_20090407.01.log
$ python 
Python 2.6 (r26:66714, Oct 16 2008, 00:21:12)
[GCC 4.2.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys, os, re, datetime, psem, psem_example
>>> psem_example.compress_log( "app1", "2009", "04", "07", "01", "bzip",
"app2_20090407.02.log");
0
>>> quit
Use quit() or Ctrl-D (i.e. EOF) to exit
>>>

Now, one can time for example 16 runs of the above command with 
16 different input files, and on a 16 CPU machine one would expect
the elapsed time between start and finish to be of the order of
1/16th of the time taken to compress all of the 16 files sequentially.
History
Date User Action Args
2009-04-09 06:43:11jvdiassetrecipients: + jvdias
2009-04-09 06:43:11jvdiassetmessageid: <1239259391.17.0.587935519406.issue5725@psf.upfronthosting.co.za>
2009-04-09 06:43:09jvdiaslinkissue5725 messages
2009-04-09 06:43:09jvdiascreate