classification
Title: Add subprocess.iter_output() convenience function
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions: Python 3.4
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: giampaolo.rodola, ncoghlan, rosslagerwall
Priority: normal Keywords:

Created on 2011-12-11 05:05 by ncoghlan, last changed 2012-06-12 12:00 by ncoghlan.

Messages (3)
msg149210 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2011-12-11 05:05
subprocess.check_output() is nice, but doesn't help if you want to process the piped data line-by-line.

Currently, that means you have to do the full Popen dance if you want access to each line of output as it becomes available.

This RFE is for a subprocess.iter_output() module level helper that:
1. Starts the subprocess
2. Yield the individual lines of output as they are produced by the subprocess
3. Cleans up (including checking for errors) at the end

This biggest challenge I have noticed so far in exploring this is how to handle timeouts on Windows - on Unix, select.select() can do the job, but that won't handle pipes in the Windows case.
msg149215 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2011-12-11 07:39
You can see a version of this here:

https://bitbucket.org/ncoghlan/shell_command/src/2b1988b072aa/shell_command.py#cl-157
msg152794 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2012-02-06 22:32
> This biggest challenge I have noticed so far in exploring 
> this is how to handle timeouts on Windows

I haven't actually looked into it but this somewhat recalls:
http://bugs.python.org/issue1191964
Since issue1191964 is supposed to provide async subprocess I/O I presume Windows timeouts are also involved somewhere in the provided patch/recipe.
History
Date User Action Args
2012-06-12 12:00:10ncoghlansetversions: + Python 3.4, - Python 3.3
2012-02-06 22:32:52giampaolo.rodolasetnosy: + giampaolo.rodola
messages: + msg152794
2011-12-11 07:39:03ncoghlansetmessages: + msg149215
2011-12-11 05:09:38rosslagerwallsetnosy: + rosslagerwall
2011-12-11 05:05:54ncoghlancreate