Author Aaron.Sherman
Recipients Aaron.Sherman, giampaolo.rodola, gregory.p.smith, neologix, pitrou, vstinner
Date 2011-03-02.21:57:42
SpamBayes Score 9.41819e-10
Marked as misclassified No
Message-id <1299103063.1.0.331245855119.issue11314@psf.upfronthosting.co.za>
In-reply-to
Content
I think it's still safe to say that high performance applications which need to create many hundreds or thousands of children (e.g. large monitoring systems) will still need another solution that isn't subprocess. That being said, you're right that no one is going to care about the extra overhead of subprocess in a vacuum, and most applications fork one or a very small number of processes at a time and typically end up waiting for orders of magnitude more time for their output than they spend creating the process in the first place.

As I said when I opened this issue, I wasn't terribly concerned with most applications.

That being said, I can't really submit a full-blown monitoring system against this bug, so the best I could do would something that did lots of os.popens or subprocess.Popens in parallel in a contrived way and see how the performance scales as I tune "lots" to different values. Sadly, the time I have for doing that needs to be spent writing other code, so I'll leave this closed and let someone else raise the issue in the future if they run into it.

I can always build a dispatcher in C and communicate with it via IPC to get around the immediate concern of scalability.
History
Date User Action Args
2011-03-02 21:57:43Aaron.Shermansetrecipients: + Aaron.Sherman, gregory.p.smith, pitrou, vstinner, giampaolo.rodola, neologix
2011-03-02 21:57:43Aaron.Shermansetmessageid: <1299103063.1.0.331245855119.issue11314@psf.upfronthosting.co.za>
2011-03-02 21:57:42Aaron.Shermanlinkissue11314 messages
2011-03-02 21:57:42Aaron.Shermancreate