Message56317
hi - I noticed some odd behaviour with subprocess when the output
(stdout) is large - I did some sleuthing and found a reproduceable case
- I tested this on solaris using python 2.4.2 (sparc) and using 2.5
(opteron) - I've attached a python script that details the problem - I
created 2 files ("small_file" and "large_file") that are 20480 and 20481
bytes, respectively ....
> ls -lrt ./tmp/*_file
-rw-r--r-- 1 wtorres staff 20481 Oct 10 12:32 ./tmp/large_file
-rw-r--r-- 1 wtorres staff 20480 Oct 10 12:32 ./tmp/small_file
the script cats each file, thus generating an appropriately sized stdout
- the workaround we came up with was to use our own file handle and use
the parameter stdout=my_file_handle.fileno() instead of stdout=PIPE -
you can see this in the attached example as well
> python ./tmp/brent.py
testing with a small file - these should both work
using my own file handle .... ok
using PIPE ... ok
testing with a large file - the second one here should fail
using my own file handle .... ok
using PIPE ...
Traceback (most recent call last):
File "./tmp/brent.py", line 43, in ?
test_popen_bug( large_file )
File "./tmp/brent.py", line 31, in test_popen_bug
rc = my_popen.wait()
File "/tools/python/2.4.2/solaris/lib/python2.4/subprocess.py", line
1007, in wait
pid, sts = os.waitpid(self.pid, 0)
KeyboardInterrupt
based on all this, it seems that the PIPE file handle has some buffer or
something that is filling up and not allowing us to continue... or
something. Thoughts?
-w |
File name |
Uploaded |
brent.py
|
superwesman,
2007-10-10.17:41:48
|
|
Date |
User |
Action |
Args |
2007-10-10 17:41:50 | superwesman | set | spambayes_score: 0.00629202 -> 0.00629202 recipients:
+ superwesman |
2007-10-10 17:41:49 | superwesman | set | spambayes_score: 0.00629202 -> 0.00629202 messageid: <1192038109.59.0.704801934631.issue1256@psf.upfronthosting.co.za> |
2007-10-10 17:41:49 | superwesman | link | issue1256 messages |
2007-10-10 17:41:48 | superwesman | create | |
|