Author neologix
Recipients gregory.p.smith, neologix, pitrou, sbt, serhiy.storchaka, vstinner
Date 2013-12-08.09:38:10
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
This is a spinoff of issue #19506: currently, subprocess.communicate() uses a 4K buffer when reading data from pipes.
This was probably optimal a couple years ago, but nowadays most operating systems have larger pipes (e.g. Linux has 64K), so we might be able to gain some performance by increasing this buffer size.

For example, here's a benchmark reading from a subprocess spawning "dd if=/dev/zero bs=1M count=100":

# before, 4K buffer
$ ./python ~/ 

# after, 64K buffer
$ ./python ~/ 

The difference is impressive.

I'm attaching the benchmark script so that others can experiment a bit (on multi-core machines and also different OSes).
Date User Action Args
2013-12-08 09:38:10neologixsetrecipients: + neologix, gregory.p.smith, pitrou, vstinner, sbt, serhiy.storchaka
2013-12-08 09:38:10neologixsetmessageid: <>
2013-12-08 09:38:10neologixlinkissue19929 messages
2013-12-08 09:38:10neologixcreate