Author superwesman
Recipients superwesman
Date 2007-10-10.17:41:48
SpamBayes Score 0.00629202
Marked as misclassified No
Message-id <1192038109.59.0.704801934631.issue1256@psf.upfronthosting.co.za>
In-reply-to
Content
hi - I noticed some odd behaviour with subprocess when the output
(stdout) is large - I did some sleuthing and found a reproduceable case
- I tested this on solaris using python 2.4.2 (sparc) and using 2.5
(opteron) - I've attached a python script that details the problem - I
created 2 files ("small_file" and "large_file") that are 20480 and 20481
bytes, respectively ....

> ls -lrt ./tmp/*_file
-rw-r--r--   1 wtorres  staff      20481 Oct 10 12:32 ./tmp/large_file
-rw-r--r--   1 wtorres  staff      20480 Oct 10 12:32 ./tmp/small_file

the script cats each file, thus generating an appropriately sized stdout
- the workaround we came up with was to use our own file handle and use
the parameter stdout=my_file_handle.fileno() instead of stdout=PIPE -
you can see this in the attached example as well

> python ./tmp/brent.py 
testing with a small file - these should both work
using my own file handle .... ok
using PIPE ... ok
testing with a large file - the second one here should fail
using my own file handle .... ok
using PIPE ...
Traceback (most recent call last):
  File "./tmp/brent.py", line 43, in ?
    test_popen_bug( large_file )
  File "./tmp/brent.py", line 31, in test_popen_bug
    rc = my_popen.wait()
  File "/tools/python/2.4.2/solaris/lib/python2.4/subprocess.py", line
1007, in wait
    pid, sts = os.waitpid(self.pid, 0)
KeyboardInterrupt

based on all this, it seems that the PIPE file handle has some buffer or
something that is filling up and not allowing us to continue... or
something.  Thoughts?
-w
Files
File name Uploaded
brent.py superwesman, 2007-10-10.17:41:48
History
Date User Action Args
2007-10-10 17:41:50superwesmansetspambayes_score: 0.00629202 -> 0.00629202
recipients: + superwesman
2007-10-10 17:41:49superwesmansetspambayes_score: 0.00629202 -> 0.00629202
messageid: <1192038109.59.0.704801934631.issue1256@psf.upfronthosting.co.za>
2007-10-10 17:41:49superwesmanlinkissue1256 messages
2007-10-10 17:41:48superwesmancreate